Effects  of  Visual,  Auditory,  and  Tactile  Navigation  Cues 
on  Navigation  Performance,  Situation  Awareness, 
and  Mental  Workload 


by  Bradley  M.  Davis 


ARL-TR-4022 


February  2007 


Approved  for  public  release;  distribution  is  unlimited. 


NOTICES 

Disclaimers 

The  findings  in  this  report  are  not  to  be  eonstrued  as  an  official  Department  of  the  Army  position 
unless  so  designated  by  other  authorized  documents. 

Citation  of  manufacturer’s  or  trade  names  does  not  constitute  an  official  endorsement  or 
approval  of  the  use  thereof. 


DESTRUCTION  NOTICE — ^Destroy  this  report  when  it  is  no  longer  needed.  Do  not  return  it  to 
the  originator. 


Army  Research  Laboratory 

Aberdeen  Proving  Ground,  MD  21005-5425 


ARL-TR-4022 


February  2007 


Effects  of  Visual,  Auditory,  and  Tactile  Navigation  Cues 
on  Navigation  Performance,  Situation  Awareness, 
and  Mental  Workload 

Bradley  M.  Davis 

Human  Research  &  Engineering  Directorate,  ARE 


Approved  for  public  release;  distribution  is  unlimited. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this 
collection  of  information,  including  suggestions  for  reducing  the  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and 
Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no 
person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1.  REPORT  DATE  (DD-MM-YYYY) 

February  2007 


4.  TITLE  AND  SUBTITLE 


2.  REPORT  TYPE 

Final 


3.  DATES  COVERED  (From  -  To) 


5a.  CONTRACT  NUMBER 


Effects  of  Visual,  Auditory,  and  Tactile  Waypoint  Location  Cues  on  Navigation 
Performance,  Situation  Awareness,  and  Mental  Workload 


5b.  GRANT  NUMBER 


5c.  PROGRAM  ELEMENT  NUMBER 


6.  AUTHOR(S) 


Bradley  M.  Davis  (ARE) 


5d.  PROJECT  NUMBER 

62716AH70 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Laboratory 

Human  Research  &  Engineering  Directorate 

Aberdeen  Proving  Ground,  MD  21005-5425 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

ARL-TR-4022 


10.  SPONSOR/MONITOR’S  ACRONYM(S) 


11.  SPONSOR/MONITOR’S  REPORT  NUMBERS 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT 

Two  experiments  were  conducted  to  assess  the  effects  of  navigation  display  modality  on  navigation  performance,  situation 
awareness,  mental  workload,  and  modality  preference.  Directional  cues  to  a  series  of  waypoints  were  provided  visually, 
aurally,  and  tactilely  in  the  within-subjects  design.  Each  experiment  was  performed  in  a  virtual  environment  by  U.S.  Army 
Soldiers,  14  in  the  first  experiment,  18  in  the  second  experiment.  Results  from  both  experiments  indicate  that  augmented 
visual  displays  reduced  time  to  complete  navigation,  maintained  situation  awareness,  and  drastically  reduced  mental  workload 
in  comparison  to  the  other  display  modalities. 


15.  SUBJECT  TERMS 

mental  workload;  navigation;  situation  awareness;  tactile 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION 
OF  ABSTRACT 

18.  NUMBER 

OF  PAGES 

19a.  NAME  OF  RESPONSIBLE  PERSON 

Bradley  M.  Davis 

a.  REPORT 

UNCLASSIFIED 

b.  ABSTRACT 

UNCLASSIFIED 

c.  THIS  PAGE 

UNCLASSIFIED 

SAR 

65 

19b.  TELEPHONE  NUMBER  (Include  area  code) 

573-329-8704 

Standard  Form  298  (Rev.  8/98) 
Prescribed  by  ANSI  Std.  Z39.18 


Contents 


List  of  Figures  v 

List  of  Tables  vii 

1.  Introduction  1 

2,  Objective  2 

3.  Experiment  1  2 

3.1  Method . 2 

3.1.1  Participants . 2 

3.1.2  Apparatus . 3 

3.1.3  Task . 4 

3.1.4  Questionnaires . 5 

3.1.5  Experimental  Design . 6 

3.1.6  Procedure . 10 

3.2  Results . 11 

3.2.1  Navigation  Performance . 11 

3 .2.2  Situation  Awareness . 14 

3.2.3  Mental  Workload . 16 

3.2.4  Modality  Preference . 18 

3.3  Discussion . 19 

3.3.1  Navigation  Performance . 20 

3.3.2  Situation  Awareness . 20 

3.3.3  Mental  Workload . 20 

3.3.4  Modality  Preference . 21 

3.3.5  Summary . 21 

4,  Experiment  2  22 

4.1  Method . 22 

4.1.1  Participants . 22 

4.1.2  Apparatus . 22 

4.1.3  Task . 22 

4.1.4  Questionnaires . 22 

iii 


4.1.5  Experimental  Design . 23 

4.1.6  Proeedure . 24 

4.2  Results . 25 

4.2.1  Navigation  Performance . 25 

4.2.2  Situation  Awareness . 28 

4.2.3  Mental  Workload . 30 

4.2.4  Modality  Preference . 31 

4.3  Discussion . 33 

4.3.1  Navigation  Performance . 33 

4.3.2  Situation  Awareness . 34 

4.3.3  Mental  Workload . 34 

4.3.4  Modality  Preference . 34 

4.3.5  Summary . 35 

5.  Conclusions  35 

6,  References  36 

Appendix  A,  Scenario  Maps  (Experiment  1)  37 

Appendix  B.  Questionnaires  41 

Appendix  C,  SART  Subscale  Score  Mean  Differences  (ANOVA post  hoc  analyses) 

(Experiment  1)  45 

Appendix  D,  NASA-TLX  Subscale  Score  Mean  Differences  (ANOVA  post  hoc  analyses) 
(Experiment  1)  47 

Appendix  E.  Scenario  Maps  (Experiment  2)  49 

Appendix  F,  SART  Subscale  Score  Mean  Differences  (ANOVA  post  hoc  analyses) 

(Experiment  2)  53 

Appendix  G,  NASA-TLX  Subscale  Score  Mean  Differences  (ANOVA  post  hoc  analyses) 
(Experiment  2)  55 

Distribution  List  56 


IV 


List  of  Figures 


Figure  1 .  Apparatus  configuration . 3 

Figure  2.  Visible  objects  at  two  of  the  waypoints  in  open  terrain . 4 

Figure  3.  Visible  object  at  the  waypoint  in  urban  terrain . 4 

Figure  4.  Minefield  marker . 5 

Figure  5.  Baseline  displays:  primary  display  (left)  and  navigation  display  (right) . 7 

Figure  6.  Close  view  of  GPS  display  area  of  the  baseline  primary  display . 7 

Figure  7.  Close  view  of  compass  display  area  of  the  baseline  primary  display . 7 

Figure  8.  Head-up  icon  displays:  close  view  of  waypoint  icon  within  FOV  (left)  and  out 

of  FOV  (right) . 8 

Figure  9.  Moving  map  displays:  primary  display  showing  GPS  and  compass  (left)  and 

waypoint  display  (right) . 8 

Figure  10.  The  MIT  WTCU  and  vibro-tactile  motors . 10 

Figure  11.  Minefield  navigation  errors  versus  waypoint  display  modality . 12 

. 12 

Figure  12.  Minefield  navigation  errors  versus  scenario  map . 12 

Figure  13.  Mean  navigation  time  versus  waypoint  display  modality . 13 

Figure  14.  Overall  SART  score  versus  waypoint  display  modality . 14 

. 15 

Figure  15.  SART  subscale  scores  versus  waypoint  display  modality . 15 

Figure  16.  Overall  NASA-TLX  score  versus  waypoint  display  modality . 16 

Figure  17.  NASA-TLX  subscale  scores  versus  waypoint  display  modality . 17 

Figure  18.  Modality  evaluation  rating  versus  waypoint  display  modality . 18 

Figure  19.  Minefield  navigation  errors  versus  waypoint  display  modality . 26 

Figure  20.  Minefield  navigation  errors  versus  waypoint  display  modality . 27 

Figure  21 .  Mean  navigation  time  versus  waypoint  display  modality . 27 

Figure  22.  Overall  SART  score  versus  waypoint  display  modality . 28 

Figure  23.  SART  subscale  scores  versus  waypoint  display  modality . 29 

Figure  24.  Overall  NASA-TLX  score  versus  waypoint  display  modality . 30 

Figure  25.  NASA-TLX  subscale  scores  versus  waypoint  display  modality . 31 

Figure  26.  Participant  evaluation  rating  versus  waypoint  display  modality . 32 

Figure  A-1 .  Map  A . 37 

Figure  A-2.  Map  B . 37 

Figure  A-3.  Map  C . 38 

Figure  A-4.  Map  D . 38 

Figure  A-5.  Map  E . 39 

Figure  A-6.  Map  F . 39 


V 


Figure  A-7.  Map  G . 40 

Figure  E-1.  Map  A . 49 

Figure  E-2.  Map  B . 49 

Eigure  E-3.  Map  C . 50 

Eigure  E-4.  Map  D . 50 

Eigure  E-5.  Map  E . 51 

Eigure  E-6.  Map  E . 51 


VI 


List  of  Tables 


Table  1.  Greco-Latin  square  design  for  experiment  1 . 10 

Table  2.  Repeated  measures  ANOVA  summary,  waypoint  display  modality  as  independent 

variable . 11 

Table  3.  Repeated  measures  ANOVA  summary,  scenario  map  as  independent  variable . 11 

Table  4.  Mean  differences  between  modalities  in  navigation  time . 14 

Table  5.  Mean  differences  between  modalities  in  SART  score . 15 

Table  6.  Repeated  measures  ANOVA  summary  of  SART  subscale  scores . 16 

Table  7.  Mean  differences  between  modalities  in  NASA-TLX  score . 17 

Table  8.  Repeated  measures  ANOVA  summary  of  NASA-  TLX  subscale  scores . 18 

Table  9.  Mean  differences  between  modalities  in  evaluation  rating . 19 

Table  10.  Inter-item  correlations  for  the  modality  evaluation  questionnaire . 19 

Table  1 1 .  Greco-Latin  square  design  for  experiment  2 . 24 

Table  12.  Repeated  measures  ANOVA  summary,  waypoint  display  modality  as  independent 

variable . 25 

Table  13.  Repeated  measures  ANOVA  summary,  scenario  map  and  order  as  independent 

variable . 25 

Table  14.  Mean  differences  between  modalities  in  navigation  time . 28 

Table  15.  Mean  differences  between  modalities  in  SART  score . 29 

Table  16.  Repeated  measures  ANOVA  summary  of  SART  subscale  scores . 29 

Table  17.  Mean  differences  between  modalities  in  NASA-TLX  score . 30 

Table  18.  Repeated  measures  ANOVA  summary  of  NASA-TLX  subscale  scores . 31 

Table  19.  Mean  differences  between  modalities  in  evaluation  rating . 32 

Table  20.  Inter-item  correlations  for  the  modality  evaluation  questionnaire . 33 

Table  C-l.  Mean  differences  between  modalities  in  SART  demand  subscale  score . 45 

Table  C-2.  Mean  differences  between  modalities  in  SART  demand  subscale  score . 45 

Table  D-1 .  Mean  differences  between  modalities  in  NASA-TLX  mental  demand  subscale 

score . 47 

Table  D-2.  Mean  differences  between  modalities  in  NASA-TLX  temporal  demand  subscale 

score . 47 

Table  D-3.  Mean  differences  between  modalities  in  NASA-TLX  effort  subscale  score . 47 

Table  D-4.  Mean  differences  between  modalities  in  NASA-TLX  frustration  subscale  score . 48 

Table  F-l.  Mean  differences  between  modalities  in  SART  demand  subscale  score . 53 

Table  G-l.  Mean  differences  between  modalities  in  NASA-TLX  mental  demand  subscale  score. 

. 55 

Table  G-2.  Mean  differences  between  modalities  in  NASA-TLX  effort  subscale  score . 55 


vii 


Intentionally  left  blank 


1.  Introduction 


This  study  is  one  of  several  investigations  in  an  iterative  cyele  of  modeling  and  experimentation  to 
achieve  the  objective  of  the  Situational  Understanding  as  an  Enabler  for  Unit  of  Action  Maneuver 
Team  Soldiers  Army  Technology  Objective  (ATO).  The  objective  of  the  ATO  was  to  develop, 
demonstrate,  and  transition  unit  of  action  Soldier  information  systems  that  address  differences  in 
the  way  Soldiers  gain  situational  understanding  and  enable  planning  and  acting  within  the 
adversary’s  decision  cycle. 

To  help  meet  the  objective  of  the  ATO,  Mitchell,  Samms,  Glumm,  Krausman,  Brelsford,  &  Garrett 
(2004)  built  Improved  Performance  Research  Integration  Tool  (IMPRINT)  models  to  simulate  tasks 
of  each  of  the  crew  positions  of  Future  Combat  Systems’  manned  ground  vehicle  (MGV)  variants. 
The  MGV  variants  modeled  were  the  Mounted  Combat  System  (MCS),  infantry  carrier  vehicle 
(ICV),  non-line  of  sight  cannon  (NLOS-C),  non-line  of  sight  mortar  (NLOS-M),  and  the  reconnais¬ 
sance  and  surveillance  vehicle  (RSV).  These  IMPRINT  analyses  have  identified  areas  of  operator 
overload.  Across  all  the  MGV  variants,  the  driver  was  often  the  crew  member  with  the  most 
instances  of  operator  overload  as  a  result  of  the  demands  of  the  tasks  associated  with  driving 
(manipulations,  awareness,  and  visualizations)  (Mitchell  et  ah,  2004).  These  driving  tasks  have 
been  validated  against  other  driving  models  and  have  been  determined  to  represent  adequately  the 
driving  function  (Wojciechowski,  2004). 

Recommendations  of  ways  to  mitigate  operator  overload  were  suggested  by  Mitchell  et  al.  (2004). 
The  recommendations  included  using  tactile  or  auditory  displays  with  the  driver’s  integrated 
display  to  reduce  the  use  of  visual  and  cognitive  resources  of  the  driver.  In  addition,  spatial  audio 
was  recommended  as  a  means  of  assisting  navigation. 

These  recommendations  stem  from  multiple  resource  theory  (MRT)  (Wickens  &  Hollands,  2000), 
which  was  also  the  basis  for  the  IMPRINT  workload  analysis.  Multiple  resource  theory  proposes 
that  people  have  several  independent  capacities  with  resource  properties  and  that  some  resources 
can  easily  be  used  simultaneously  while  other  combinations  are  much  more  difficult.  Therefore, 
tasks  using  compatible  resources  can  usually  be  performed  together,  while  competition  for  the 
same  modality  can  produce  interference.  The  degree  of  task  compatibility  or  interference  affects 
the  level  of  performance. 

This  study  focused  on  the  driver’s  task  of  navigation.  Navigation  is  a  process  performed  by  the 
driver,  which  includes  all  the  driving  tasks:  manipulation,  awareness,  and  visualization.  These 
tasks  are  primarily  visually  and  cognitively  oriented.  A  fundamental  element  of  navigation  is 
having  the  knowledge  of  one’s  current  location  and  the  location  of  the  destination,  which  may  be 
presented  to  the  driver  in  a  variety  of  ways. 
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From  the  recommendations  by  Mitchell  et  al.  (2004),  which  are  supported  by  MRT,  several 
display  modalities  for  presenting  navigation  waypoint  information  were  designed,  including 
visual,  monaural  and  spatial  audio,  and  tactile  displays.  A  comparison  of  these  displays  and  a 
baseline  display  in  terms  of  navigation  performance,  situation  awareness  (SA),  and  mental  work¬ 
load  was  conducted.  It  was  hypothesized  that  the  alternate  display  modalities  would  provide 
greater  navigation  performance,  increased  SA,  and  decreased  mental  workload  than  the  baseline 
display  configuration  because  of  the  improved  information  processing  provided  by  the  displays.  It 
was  not  hypothesized  which  specific  waypoint  display  modality  would  provide  best  performance, 
SA,  or  mental  workload  but  that  there  would  be  significant  differences  among  the  different 
experimental  conditions  for  each  dependent  variable. 


2.  Objective 


The  objective  of  this  study  was  to  assess  the  effects  of  waypoint  display  modality  on  navigation 
performance,  SA,  mental  workload,  and  modality  preference.  To  reach  this  objective,  two 
experiments  were  conducted:  the  first  laid  the  groundwork  for  the  study  and  the  second  refined 
the  methods  and  validated  the  results  of  the  first  experiment. 


3.  Experiment  1 


3.1  Method 

3.1.1  Participants 

The  participants  for  the  first  experiment  were  14  male  U.S.  Army  Soldiers,  ranging  in  age  from 
20  to  27  years  (mean  [M]  =  22.0  years,  standard  deviation  [SD]  =  2.25  years).  Participants  had 
from  0.7  to  3.6  years  of  military  service  (M=  2.21  years,  SD  =  0.7  years)  and  a  similar  amount  of 
time  in  their  military  occupational  specialty  (MOS).  All  participants  were  armor  crewmen,  MOS 
19K,  or  cavalry  scout,  MOS  19D  and  were  qualified  as  drivers  of  the  Ml  Abrams  tank.  All  the 
participants  had  normal  visual  acuity  and  color  vision  in  both  eyes.  Although  participants  were 
not  given  a  hearing  threshold  level  test,  all  participants  reported  no  suspected  hearing  loss  or 
damage.  The  voluntary,  fully  informed  consent  of  the  persons  used  in  this  research  was  obtained 
as  required  by  32  Code  of  Federal  Regulations  (CFR)  219  and  Army  Regulation  (AR)  70-25. 

The  investigator  has  adhered  to  the  policies  for  the  protection  of  human  subjects  as  prescribed  in 
AR  70-25  (HQDA,  1990). 
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3,1,2  Apparatus 

The  participants  sat  at  a  computer  workstation  with  two  personal  computers  (PCs),  each  equipped 
with  a  3.0-GHz  Intel  Pentium  4  central  processing  unit,  ATP  Radeon^  9800  Pro  256  MB  graphics 
adapter,  and  1  GB  of  random  access  memory  (RAM).  A  ViewSonic  VG900b  19-inch  liquid 
crystal  display  (LCD)  monitor  set  at  1024  x  768  resolution  was  connected  to  each  PC  as  a  primary 
display  and  navigation  display.  The  primary  display  was  placed  at  a  comfortable  distance,  directly 
in  front  of  the  participant.  The  navigation  display  was  placed  immediately  to  the  right  of  the 
primary  display  and  angled  toward  the  participant.  The  navigation  display  was  only  used  for  two 
of  the  experimental  conditions  and  was  turned  off  for  all  other  experimental  conditions.  A  Cyber 
Acoustics  AC-200  supra-aural  stereo  headset  was  connected  to  the  PC  via  the  Creative  Labs  Sound 
Blaster  Audigy^  sound  card.  The  input  devices  used  for  navigation  were  a  Microsoft  optical  wheel 
mouse  and  Microsoft  multimedia  keyboard.  An  additional  Microsoft  optical  wheel  mouse  served 
as  the  input  device  for  the  navigation  display  for  the  two  experimental  conditions  requiring  the 
navigation  display  and  was  moved  out  of  the  way  for  all  other  experimental  conditions.  Figure  1 
graphically  illustrates  the  computer  workstation  configuration. 


Figure  1.  Apparatus  configuration. 


'ATI  is  not  an  acronym. 

'y 

Radeon  is  a  registered  trademark  of  ATI  Technologies,  Inc. 

Blaster  and  Audigy  are  registered  trademarks  of  Creative  Technology  Ltd. 
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VBSl"^  was  used  as  the  virtual  environment  for  these  experiments.  VBSl  is  a  military  training  and 
simulation  system  used  by  the  U.S.  Marine  Corps  and  the  U.S.  Army  National  Guard.  VBSl  is  an 
immersive  three-dimensional  first-person  shooter-style  simulation,  with  the  additional  ability  to 
eommand  and  control  many  vehicles  and  aircraft.  The  simulation  allows  for  cooperative  or  com¬ 
petitive  multi-player  scenarios.  Navigation  scenarios  were  created  with  the  VBSl  scenario  editor 
which  allows  for  complex  scenario  editing,  custom  scripting,  and  a  means  for  external  file  input 
and  output,  allowing  simple  integration  of  the  tactile  equipment  used. 

3,1,3  Task 

Each  navigation  scenario  consisted  of  a  series  of  three  waypoints.  Each  waypoint  included  a 
visible  object,  such  as  an  abandoned  tank  (see  figure  2),  to  assist  the  participant  in  identifying  the 
waypoint  and  include  an  aspect  of  realism.  Two  of  the  waypoints  were  in  open,  rolling,  desert 
terrain.  Another  navigation  segment  placed  the  waypoint  in  an  urban  area  (see  figure  3).  The 
first  navigation  segment  was  1  km  long,  and  the  second  and  third  segments  were  2  km  each.  The 
locations  of  the  waypoints  were  randomized  so  that  a  participant  would  not  learn  and  memorize 
the  navigation  route.  Randomly,  one  of  the  three  navigation  segments  included  identifying  and 
navigating  around  a  200-m  by  1000-m  minefield,  marked  with  danger  signs  around  the  perimeter 
(see  figure  4).  Diagrams  of  the  scenario  maps  are  provided  in  appendix  A. 


Figure  2.  Visible  objeets  at  two  of  the  waypoints  in  open  terrain. 


Figure  3.  Visible  objeet  at  the  waypoint  in  urban  terrain. 


^VBSl  is  a  trademark  of  Bohemia  Interaetive  Studio. 
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Figure  4.  Minefield  marker. 


3,1,4  Questionnaires 

Several  questionnaires  were  used  as  measures  of  the  dependent  variables.  Each  is  described  in 
more  detail  next. 

3 . 1 .4. 1  Situational  Awareness  Rating  Technique  (SART) 

SA  was  measured  with  the  SART  questionnaire.  The  metric  combines  10  generic  constructs 
derived  from  knowledge  elicitation  techniques  with  air  crews  into  three  broad  domains:  (a)  atten- 
tional  demand,  which  includes  constructs  of  instability  of  situation,  variability  of  situation,  and 
complexity  of  situation;  (b)  attentional  supply,  which  includes  the  constructs  of  arousal,  spare 
mental  capacity,  concentration,  and  division  of  attention;  and  (c)  understanding,  which  includes  the 
constructs  of  information  quantity,  information  quality,  and  familiarity  (Taylor,  1990).  These  three 
broad  domains  are  combined  to  form  an  overall  SART  score  by  the  formula  SA  =  Understanding  - 
(Demand  -  Supply).  A  high  overall  SART  score  indicates  a  high  (better)  level  of  SA,  whereas  a 
low  SART  score  indicates  a  low  (worse)  level  of  SA.  The  SART  questionnaire  used  in  these 
experiments  is  shown  in  appendix  B. 

3. 1.4.2  National  Aeronautics  and  Space  Administration  (NASA)  Task  Load  Index  (TLX) 

Mental  workload  was  measured  with  a  NASA-TLX  questionnaire.  The  NASA- TLX  is  a  psycho¬ 
physical  technique  for  mental  workload  measurement,  which  has  sensitivity  to  different  levels  of 
task  demand.  The  NASA-TLX  provides  an  overall  subjective  workload  score  based  on  a  weighted 
average  of  ratings  on  six  subscales:  mental  demand,  physical  demand,  temporal  demand,  own 
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performance,  effort,  and  frustration  (Hart  &  Staveland,  1988).  For  these  experiments,  the  physical 
demand  subscale  was  removed  because  of  the  non-physical  nature  of  the  task.  After  each  subscale 
is  rated,  the  participants  select  from  combinations  of  paired  comparisons,  the  more  important  con¬ 
tributor  to  workload  for  the  task  performed  to  create  a  weighting  for  each  subscale.  The  weighted 
ratings  are  then  combined  to  create  an  overall  NASA- TLX  score.  A  high  NASA-TLX  score 
indicates  a  high  level  of  mental  workload,  and  a  low  NASA-TLX  score  indicates  a  low  level  of 
mental  workload.  The  NASA-TLX  questionnaire  used  in  these  experiments  is  included  in 
appendix  B. 

3. 1.4. 3  Modality  Preference 

To  assess  modality  preference,  a  modality  evaluation  questionnaire  was  designed  to  allow  the 
participant  to  subjectively  rate  each  experimental  condition.  The  questionnaire  consisted  of  seven 
question  items  using  a  five-point  Likert  type  scale  (1  =  strongly  disagree;  5  =  strongly  agree), 
except  for  one  negatively  scored  question  item  in  which  the  scale  values  were  reversed.  We 
created  an  overall  modality  rating  score  by  combining  the  seven  question  items  and  averaging  the 
ratings  from  each  item.  The  modality  evaluation  questionnaire  is  included  in  appendix  B. 

3.1.5  Experimental  Design 

Experiment  1  was  a  single  factor  within-subjects  design.  The  independent  variable  was  waypoint 
display  modality  and  the  dependent  variables  were  navigation  performance,  SA,  mental  workload, 
and  modality  preference.  Each  dependent  variable  and  its  associated  measures  are  described  in 
more  detail  following. 

3 . 1 .5 . 1  Navigation  Performance 

Two  metrics  were  used  for  navigation  performance:  minefield  navigation  errors  and  navigation 
time.  A  minefield  navigation  error  was  recorded  when  a  participant  accidentally  breached  the 
minefield.  A  30-second  time  penalty  was  assessed  for  breaching  a  minefield  since  doing  so 
reduced  the  total  distance  traveled,  thus  decreasing  the  time  elapsed  between  waypoints.  Navi¬ 
gation  time  was  recorded  as  the  total  time  elapsed  from  the  starting  location  until  the  participant 
reached  the  final  waypoint,  including  any  time  spent  planning  the  route  and  any  time  penalties 
attributable  to  minefield  navigation  errors. 

3 . 1 .5 .2  Situation  Awareness 

The  overall  score  from  the  SART  questionnaire  was  used  to  measure  SA. 

3. 1.5. 3  Mental  Workload 

The  overall  score  from  the  NASA-TLX  questionnaire  was  used  to  measure  mental  workload. 
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3. 1.5.4  Modality  Preference 

The  overall  score  from  the  modality  rating  questionnaire  was  used  to  measure  modality  preference. 


3. 1.5. 5  Seven  Levels  of  Waypoint  Display  Modality 

There  were  seven  levels  of  the  waypoint  display  modality  independent  variable:  baseline,  head-up 
icon,  moving  map,  monaural  speech,  spatial  speech,  spatial  audio  tone,  and  tactile.  Each  con-dition 
is  described  in  more  detail  following. 


•  Baseline  -  A  static,  two-dimensional  (2-D)  overhead  map  provided  on  the  navigation 
display  showed  the  starting  position,  waypoint  locations,  global  positioning  system  (GPS) 
grid  lines,  terrain  features,  and  landmarks  but  did  not  update  or  show  the  location  or 
heading  of  the  vehicle  (see  figure  5).  The  participant  used  the  primary  display  along  with 
the  GPS  display  (in  standard  universal  transverse  mercator  [UTM]  format)  and  compass  to 
determine  his  current  location  and  heading  to  the  next  waypoint.  Close  views  of  the  GPS 
display  and  compass  display  areas  are  presented  in  figures  6  and  7,  respectively. 


Figure  5.  Baseline  displays:  primary  display  (left)  and  navigation  display  (right). 


UTM:  38  S  MC19531530^^^, _ 

_ r 


Figure  6.  Close  view  of  GPS  display  area  of  the  baseline  primary  display. 
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Figure  7.  Close  view  of  compass  display  area  of  the  baseline  primary  display. 
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•  Head-up  icon  -  A  yellow,  square  icon  was  overlaid  on  the  primary  display,  indicating  the 
location  of  the  waypoint  (see  figure  8).  When  the  waypoint  was  out  of  the  driver’s  field  of 
view  (FOV),  the  icon  changed  to  a  triangular  arrow,  indicating  the  direction  toward  the 
waypoint.  The  word  “waypoint”  and  the  distance  to  the  waypoint  were  shown  next  to  the 
icon. 


Figure  8.  Ftead-up  icon  displays:  close  view  of  waypoint  icon  within  FOV  (left)  and  out  of  FOV  (right). 


•  Moving  map  -  A  2-D  overhead  map  provided  on  the  navigation  display  showed  the 
starting  position,  waypoint  locations,  GPS  coordinates,  terrain  features,  and  landmarks, 
exactly  the  same  as  the  baseline  condition  but  also  showed  the  location  of  the  vehicle 
updated  in  real  time  (see  figure  9).  The  vehicle  icon  was  at  the  center  of  the  map  and  was 
rotated  to  show  heading,  while  the  map  translated  up/down/left/right  to  show  position. 


Figure  9.  Moving  map  displays:  primary  display  showing  GPS  and  compass  (left)  and  waypoint  display  (right). 


•  Monaural  speech  -  Each  100  meters  traveled,  a  synthetic  speech  voice  (generated  with  the 
AT&T  Natural  Voices  Text-to-Speech  Engine)  announced  the  relative  direction  to  the  next 
waypoint  (e.g.,  “waypoint  at  10  o’clock”).  Distance  to  the  next  waypoint  was 
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communicated  by  the  same  synthetie  speeeh  voiee  at  designated  distanee  intervals  of  1000, 
500,  400,  300,  200,  and  100  meters  from  the  waypoint  (e.g.,  “400  meters  to  waypoint”). 

The  heading  and  distanee  information  was  presented  at  equal  loudness  to  both  ears. 

•  Spatial  speech  -  Emersys  Maven  3D  Professional  was  used  to  ereate  spatial  audio  sound 
representa-tions  of  the  same  synthetie  speeeh  voiee  used  in  the  monaural  speeeh  eondition. 
The  software  uses  a  generie  head-related  transform  funetion  to  produee  a  stereo  sound 
represen-tation  of  monaural  sounds.  Eaeh  100  meters  traveled,  the  synthetie  speeeh  voiee 
announeed  the  relative  direetion  to  the  next  waypoint,  whieh  appeared  to  emanate  from  the 
direetion  of  the  next  waypoint  (e.g.,  “waypoint  at  1  o’eloek”  is  heard  from  the  30-degree 
azimuth).  Distanee  was  presented  in  the  same  manner  as  the  monaural  speeeh  eondition. 

•  Spatial  audio  tone  -  Emersys  Maven  3D  Professional  was  used  to  ereate  spatial  audio 
representations  of  an  audio  tone.  Eaeh  100  meters  traveled,  an  intermittent  (600  ms  on,  400 
ms  off)  420-Hz  pure  tone  was  presented,  emanating  from  the  direetion  of  the  next  waypoint 
(e.g.,  if  the  next  waypoint  was  at  2  o’eloek,  the  audio  tone  emanated  from  the  60-degree 
azimuth).  Tones  emanating  from  the  4  to  8  o’eloek  direetions  were  redueed  in  frequeney  by 
100  Hz  to  reduee  eonfusion  of  the  mirrored  tones  about  the  3-to-9-o’eloek  axis.  Distanee 
was  presented  in  the  same  manner  as  the  monaural  speeeh  modality. 

•  Tactile  -  The  Massaehusetts  Institute  of  Teehnology  (MIT)  wireless  taetile  eontrol  unit 
(WTCU)  (see  figure  10)  was  used  with  12  vibro-taetile  motors  positioned  at  the  12  eloek 
positions  in  a  belt  worn  around  the  partieipant’s  abdomen.  Eaeh  100  meters  traveled,  the 
WTCU  provided  two  1000-ms  bursts  with  a  333-ms  rest  in  the  relative  direetion  to  the  next 
waypoint  (e.g.,  if  the  next  waypoint  was  at  3  o’eloek,  the  stimulus  vibrated  at  the 
partieipant’s  right  side).  The  vibro-taetile  motors  vibrated  at  approximately  70  Hz.  Distanee 
was  presented  in  the  same  manner  as  the  monaural  speeeh  modality. 

The  navigation  display  was  only  aetivated  (i.e.,  powered  on)  for  the  baseline  and  moving  map 
eonditions.  Eor  all  the  eonditions,  when  a  waypoint  was  reaehed,  the  synthetie  speeeh  voiee 
announeed  “waypoint  reaehed”  to  make  elear  to  the  partieipant  that  he  had  reaehed  the  waypoint. 
All  the  generated  sound  files  used  (synthetie  speeeh,  spatial  synthetie  speeeh,  and  spatial  audio 
tones)  were  normalized  to  98%  of  maximum  volume  with  Adobe  Audition^. 

The  author  ereated  a  Greeo-Latin  square  design  by  superimposing  two  balaneed  Eatin  squares, 
one  for  display  modality  and  one  for  seenario  map  (see  table  1).  The  number  and  letter 
eombination  in  eaeh  eell  of  table  1  indieates  the  seenario  that  was  used  for  eaeh  partieipant; 
display  modality  (1-7)  is  shown  first  and  seenario  map  (A-G)  is  shown  seeond. 


^Adobe  and  Audition  are  registered  trademarks  of  Adobe  Systems,  Ine. 
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Figure  10.  The  MIT  WTCU  and  vibro-tactile  motors. 


Table  1.  Greco-Latin  square  design  for  experiment  1 


Order 

1st 

2nd 

3rd 

4th 

5th 

6th 

7th 

Participant 

Number 

1,8 

IB 

2C 

7A 

3D 

6G 

4E 

5F 

2,9 

2A 

3B 

IG 

4C 

7F 

5D 

6E 

3, 10 

3G 

4A 

2F 

5B 

IE 

6C 

7D 

4,11 

4F 

5G 

3E 

6A 

2D 

7B 

1C 

5, 12 

5E 

6F 

4D 

7G 

3C 

lA 

2B 

6, 13 

6D 

7E 

5C 

IF 

4B 

2G 

3A 

7, 14 

7C 

ID 

6B 

2E 

5A 

3F 

4G 

3,1,6  Procedure 

Volunteers  received  an  overview  of  the  experiment  and  details  of  the  procedures  and  were 
informed  of  any  risks  involved  in  their  participation.  The  volunteers  read  aloud  and  were  asked 
to  sign  an  informed  consent  form  if  they  agreed  to  participate.  After  their  fully  informed, 
voluntary  consent  was  obtained,  participants  completed  a  demographics  questionnaire,  visual 
acuity  test,  and  a  color  deficiency  test. 

Each  participant  received  2  hours’  training  and  practice  with  VBSl.  First,  participants  were 
familiarized  with  VBSl,  focusing  on  the  controls  via  the  keyboard  and  mouse.  Participants  then 
completed  practice  scenarios  for  each  of  the  seven  experimental  conditions,  each  lasting 
approximately  10  minutes.  Participants  also  received  instructions  and  practiced  completing  the 
SART,  NASA-TLX,  and  modality  evaluation  questioimaires  after  the  final  practice  scenario. 
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After  training  was  complete,  participants  completed  the  seven  experimental  conditions.  Before 
each  scenario,  the  participants  received  a  short  briefing  reminding  them  of  the  visible  markers  at 
the  waypoints,  were  told  to  be  aware  of  and  navigate  around  the  minefield,  and  were  told  to  reach 
each  of  the  waypoints  as  quickly  as  possible.  After  completing  each  experimental  condition,  the 
participant  completed  the  SART,  NASA- TLX,  and  modality  evaluation  questionnaires. 


3,2  Results 

SPSS^  for  Windows^,  Release  13,  was  used  for  statistical  analysis.  Each  dependent  variable  was 
analyzed  for  significant  differences  between  waypoint  display  modalities  via  a  repeated  measures 
analysis  of  variance  (ANOVA).  Additionally,  separate  repeated  measures  ANOVAs  were  con¬ 
ducted  with  presentation  order  and  scenario  map  as  the  independent  variable.  When  significant 
differences  were  present  (a  <  .05),  post  hoc  tests  were  performed  with  the  least  significant 
difference  (LSD)  method.  There  were  significant  effects  of  waypoint  display  modality  and  a 
significant  effect  of  scenario  map  but  no  significant  effects  of  order  on  any  of  the  dependent 
variables.  A  summary  of  the  repeated  measures  ANOVA  results  for  modality  and  scenario  map  is 
presented  in  tables  2  and  3,  respectively. 


Table  2.  Repeated  measures  ANOVA  summary,  waypoint  display  modality 
as  independent  variable. 


Dependent  Variable 

F  (6,  78) 

Significance 

Minefield  Navigation  Error 

1.099 

p  =  .371 

Navigation  Time 

6.175 

p  <  .001 

Situation  Awareness  (SART) 

5.496 

p  <  .001 

Mental  Workload  (NASA-TLX) 

7.445 

p  <  .001 

Modality  Preference 

14.603 

p  <  .001 

Table  3.  Repeated  measures  ANOVA  summary,  seenario  map  as 
independent  variable. 


Dependent  Variable 

F  (6,  78) 

Significance 

Minefield  Navigation  Error 

2.241 

p  =  .049 

Navigation  Time 

0.971 

p  =  .450 

Situation  Awareness  (SART) 

0.403 

II 

bo 

Mental  Workload  (NASA-TLX) 

0.826 

p  =  .553 

Modality  Preference 

0.882 

p  =  .513 

3.2,1  Navigation  Performance 

Two  metrics  were  used  for  navigation  performance:  minefield  navigation  errors  and  navigation 
time.  The  percentage  of  minefield  navigation  errors  versus  waypoint  display  modality  is  shown 
in  figure  1 1 .  From  the  repeated  measures  ANOVA,  no  significant  differences  between  waypoint 
display  modalities  for  minefield  navigation  error  were  identified. 


®SPSS,  which  stands  for  Statistical  Package  for  the  Social  Sciences,  is  a  registered  trademark  of  SPSS,  Inc. 
^Windows  is  a  registered  trademark  of  Microsoft  Corporation. 
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Significant  differences  were  identified  between  seenario  maps  for  minefield  navigation  errors,  F 
(6,  78)  =  2.241, =  .049.  Figure  12  presents  the  pereentage  of  minefield  navigation  errors  versus 
scenario  map.  Post  hoc  analysis  indicated  that  minefield  navigation  errors  were  greater  for 
scenario  map  F  than  for  scenario  maps  A  and  B  (all  p  <  .05).  Similarly,  minefield  navigation 
errors  were  greater  for  seenario  map  E  than  for  seenario  map  B  (/?  <  .05).  Signifieant  differenees 
did  not  exist  among  scenario  map  A,  B,  C,  D,  or  G  (all p  >  .05). 


Figure  11.  Minefield  navigation  errors  versus  waypoint  display  modality. 


Figure  12.  Minefield  navigation  errors  versus  scenario  map. 
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In  order  to  conduct  the  repeated  measures  ANOVA  for  navigation  time,  missing  values  had  to  be 
computed  because  of  those  scenarios  that  were  not  completed  successfully  (e.g.,  a  fatal  minefield 
navigation  error  was  committed,  thus  ending  the  scenario).  Four  of  a  total  of  98  scenarios  (4.08%) 
had  missing  values  which  had  to  be  replaced.  Missing  data  were  replaced  with  the  series  mean  for 
each  waypoint  modality.  Mean  navigation  time  versus  waypoint  display  modality  is  presented  in 
figure  13. 
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Figure  13.  Mean  navigation  time  versus  waypoint  display  modality.  (Error 
bars  show  95%  confidence  interval.) 


Significant  differences  existed  between  waypoint  display  modalities  for  navigation  time, 

F  (6,  78)  =  6.\15,p  <  .001 .  Post  hoc  analysis  indicated  that  total  navigation  time  was  greater  in 
the  baseline  modality  than  for  the  head-up  icon,  moving  map,  and  monaural  speech  waypoint 
display  modalities  (all  p  <  .05).  Likewise,  navigation  time  in  the  spatial  audio  tone  modality  was 
greater  than  the  head-up  icon,  moving  map,  and  monaural  speech  waypoint  display  modalities  (all 
p  <  .05).  Significant  differences  did  not  exist  between  the  head-up  icon,  moving  map,  or  monaural 
speech  waypoint  display  modalities  (all  p  >  .05)  or  between  the  baseline,  spatial  speech,  spatial 
tone,  or  tactile  waypoint  display  modalities  (all  p  >  .05).  A  summary  of  navigation  time  mean 
differences  is  presented  in  table  4. 
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Table  4.  Mean  differences  between  modalities  in  navigation  time. 


Modality 

Head-up 

Icon 

Moving 

Map 

Monaural 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Tactile 

Baseline 

20I.7I4 

(p  =  .001) 

205.571 

(p  =  .001) 

136.214 
(p  =  .029) 

89.143 

(b=.124) 

-7.714 
ip=  .920) 

99.000 

(p=.108) 

Head-up  Icon 

3.857 

(P=.821) 

-65.500 

(p=.071) 

-112.571 
(p  =  .005) 

-209.429 

(p  =  .002) 

-102.714 

(p  =  .006) 

Moving  Map 

-69.357 

(p  =  .061) 

-116.429 
(p  =  .004) 

-213.286 

{p  =  .002) 

-106.571 
(p  =  .019) 

Monaural 

Speech 

-47.071 
(p  =  .309) 

-143.929 
(p  =  .032) 

-37.214 
(7^  =.418) 

Spatial  Speech 

-96.857 
(P  =  .n9) 

9.857 

(p=.790) 

Spatial  Tone 

106.714 

(p=.149) 

Bold  blocks  indicate  significant  differences. 


3,2,2  Situation  Awareness 

Mean  overall  SART  and  SART  subseale  scores  versus  waypoint  display  modality  are  presented  in 
figures  14  and  15,  respectively.  Significant  differences  were  found  between  waypoint  display 
modalities  for  overall  SART  score,  F  (6,  78)  =  5.496, <  .001.  Post  hoc  analysis  indicated  that 
overall  SART  scores  were  greater  (indicating  a  higher  level  of  SA)  in  the  moving  map  modality 
than  the  monaural  speech,  spatial  speech,  spatial  tone,  and  tactile  waypoint  display  modalities  (all 
p  <  .05).  Overall  SART  scores  were  lower  in  the  spatial  audio  tone  modality  than  all  other 
waypoint  display  modalities  (all p  <  .05).  Significant  differences  did  not  exist  between  the  base¬ 
line,  head-up  icon,  or  moving  map  waypoint  display  modalities  (all  p  >  .05).  There  were  no  signi¬ 
ficant  differences  between  monaural  speech  and  spatial  speech  waypoint  display  modalities 
ip  =  .372).  A  summary  of  SART  score  mean  differences  is  presented  in  table  5. 
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Figure  14.  Overall  SART  seore  versus  waypoint  display  modality.  (Error 
bars  show  95%  confidence  interval.) 
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Figure  15.  SART  subscale  scores  versus  waypoint  display  modality.  (Error  bars 
show  95%  confidence  interval.) 

Table  5.  Mean  differences  between  modalities  in  SART  score. 


Modality 

Head-up 

Icon 

Moving 

Map 

Monaural 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Tactile 

Baseline 

-4.929 

{p  =  .221) 

-7.429 

(p=.059) 

2.286 

(p=.453) 

5.500 
(p  =  .303) 

13.286 
(p  =  .036) 

4.143 

(p=.405) 

Head-up  Icon 

-2.500 

{p  =  .200) 

7.214 

(p=.059) 

10.429 
(p  =  .065) 

18.214 
(p  =  .009) 

9.071 

(;^=.134) 

Moving  Map 

_  _  _  _ 

-  . . . 

9.714 
{p  =  .005) 

12.929 
(P  =  .oil) 

20.714 

{p  =  .002) 

11.571 
{p  =  .032) 

Monaural 

Speech 

3.214 

(p=.327) 

11.000 
{p  =  .014) 

1.857 

(p=.546) 

Spatial  Speech 

7.786 
(p  =  .003) 

-1.357 

(p=.674) 

Spatial  Tone 

-9.143 
(p  =  .003) 

Bold  blocks  indicate  significant  differences. 


Figure  15  illustrates  the  three  SART  subscale  scores:  demand,  supply,  and  understanding.  The 
repeated  measures  ANOVA  revealed  significant  differences  in  the  demand  and  understanding 
subscales  but  not  the  supply  subscale.  The  results  of  the  ANOVA  are  summarized  in  table  6. 
Results  of  post  hoc  analyses  of  SART  subscale  scores  with  significant  differences  are  included  in 
appendix  C. 
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Table  6.  Repeated  measures  ANOVA  summary  of  SART 
subseale  seores. 


SART  Subscale 

F  (6,  78) 

Sig. 

Demand 

4.444 

o 

p 

II 

Supply 

1.793 

p^.in 

Understanding 

5.594 

p<.m\ 

3,2,3  Mental  Workload 

Mean  overall  NASA-TLX  seore  versus  waypoint  display  modality  is  presented  in  figure  16. 
Signifieant  differences  were  found  between  waypoint  display  modalities  for  overall  NASA-TLX 
score,  F  (6,  78)  =  7.445, <  .001.  Post  hoc  analysis  indicated  that  overall  NASA-TLX  scores  were 
greater  in  the  spatial  tone  modality  than  in  the  head-up  icon,  moving  map,  monaural  speech,  spatial 
speech,  and  tactile  waypoint  display  modalities  (all  p  <  .01).  Overall  NASA-TLX  scores  were 
lower  in  both  the  head-up  icon  and  moving  map  modalities  than  the  baseline,  monaural  speech,  and 
spatial  tone  waypoint  display  modalities  (all p  <  .05).  Significant  differences  did  not  exist  between 
the  baseline,  monaural  speech,  spatial  speech,  or  tactile  waypoint  display  modali-ties  (all p  >  .05). 
A  summary  of  NASA-TLX  score  mean  differences  is  presented  in  table  7. 
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Figure  16.  Overall  NASA-TLX  score  versus  waypoint  display  modality.  (Error 
bars  show  95%  confidence  interval.) 
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Table  7.  Mean  differenees  between  modalities  in  NASA-TLX  score. 


Bold  blocks  indicate  significant  differences. 


Figure  17  illustrates  the  NASA-TLX  subseale  seores;  mental  demand,  temporal  demand,  per- 
formanee,  effort,  and  frustration.  An  ANOVA  revealed  signifieant  differenees  in  all  these  sub- 
seales,  exeept  performanee.  The  ANOVA  results  are  summarized  in  table  8.  Post  hoc  analyses 
of  NASA-TLX  subseale  seores  with  signifieant  differenees  are  provided  in  appendix  D. 
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Figure  17.  NASA-TLX  subscale  scores  versus  waypoint  display  modality. 
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Table  8.  Repeated  measures  ANOVA  summary  of  NASA- 
TLX  subscale  scores. 


NASA-TLX  Subscale 

F  (6,  78) 

Sig. 

Mental  Demand 

7.351 

p<.001 

Temporal  Demand 

3.675 

p  =  .003 

Performance 

1.864 

p  =  .098 

Effort 

6.333 

p<.001 

Frustration 

8.850 

p<.001 

3,2,4  Modality  Preference 

Mean  overall  rating  versus  waypoint  display  modality  for  the  modality  evaluation  questionnaire  is 
presented  in  figure  18.  Significant  differences  exist  between  waypoint  display  modalities  for 
evaluation  rating,  F  (6,  78)  =  14.603, <  .001.  Post  hoc  analysis  indicated  that  evaluation  ratings 
were  lower  in  the  spatial  tone  modality  than  all  other  waypoint  display  modalities  (all  p  <  .01). 
Evaluation  ratings  were  higher  for  the  moving  map  modality  than  for  the  baseline,  spatial  speech, 
spatial  tone,  and  tactile  waypoint  display  modalities  (all  p  <  .05).  Significant  differences  did  not 
exist  between  the  head-up  icon  and  moving  map  display  modalities  or  the  monaural  speech  and 
spatial  speech  waypoint  display  modalities  (all  p  >  .25).  A  summary  of  evaluation  rating  mean 
differences  is  presented  in  table  9. 
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Figure  18.  Modality  evaluation  rating  versus  waypoint  display  modality.  (Error 
bars  show  95%  confidence  interval.) 
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Table  9.  Mean  differences  between  modalities  in  evaluation  rating. 


Modality 

Head-up 

Icon 

Moving 

Map 

Monaural 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Tactile 

Baseline 

-0.714 
(P  =  .010) 

-0.867 

(p  <  .001) 

-0.520 
(P  =  .011) 

-0.429 

(p=.057) 

1.153 

(p  =  .008) 

-0.194 

(P=.517) 

Head-  up  Icon 

-0.153 
(p  =  .254) 

0.194 
{p  =  .434) 

0.286 
ip- 273) 

1.867 

(p  <  .001) 

0.520 
(p=  .109) 

Moving  Map 

0.347 

(p  =  .066) 

0.439 
(p  =  .015) 

2.020 
(p  <  .001) 

0.673 
(p  =  .013) 

Monaural 

Speech 

0.092 

(p=.553) 

1.673 

(p  <  .001) 

0.327 

(P-.192) 

Spatial  Speech 

1.582 

(p  <  .001) 

0.235 

(P=.195) 

Spatial  Tone 

-1.347 

(p  <  .001) 

Bold  blocks  indicate  significant  differences. 


To  further  analyze  the  modality  evaluation  questionnaire,  inter-item  eorrelations  (see  table  10) 
and  Chronbaeh’s  alpha  were  eomputed.  Chronbaeh’s  alpha  was  ealeulated  to  be  0.933  for  the 
seale. 


Table  10.  Inter-item  correlations  for  the  modality  evaluation  questionnaire. 


Question  Item 

Efficient 

Simple 

Noticed 

Attention 

Helpful 

Annoying 

Effective 

.891 

.899 

.703 

.590 

.823 

.494 

Efficient 

.920 

.661 

.591 

.753 

.445 

Simple 

.710 

.639 

.824 

.450 

Noticed 

.849 

.771 

.425 

Attention 

.745 

.366 

Helpful 

.571 

All  correlations  significant  at  the  0.01  level  (2 -tailed). 


3,3  Discussion 

The  hypothesis  that  the  alternate  display  modalities  would  provide  greater  navigation  performanee, 
inereased  SA,  and  deereased  mental  workload  over  the  baseline  display  eonfiguration  is  not  fully 
supported  by  the  results.  There  were  no  signifioant  differenees  between  the  baseline  and  some  of 
the  alternate  modalities  for  the  navigation  performanee  and  mental  workload  dependent  variables. 
For  SA,  an  alternate  modality  (spatial  tone)  resulted  in  lower  SA  than  the  baseline  modality, 
direetly  refuting  the  hypothesis. 

The  results  of  experiment  1  support  the  hypothesis  that  there  would  be  signifieant  differenees 
between  waypoint  display  modalities.  Exeept  for  minefield  navigation  errors,  all  the  dependent 
variables  exhibited  signifieant  differenees  between  display  modalities.  Results  of  the  repeated 
measures  ANOVA  showed  that  minefield  errors  were  affeeted  by  seenario  map,  but  no  other 
variables  were  affeeted  by  seenario  map.  Presentation  order  did  not  exhibit  any  signifieant  effeet 
on  any  of  the  dependent  variables. 
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3.3.1  Navigation  Performance 

There  were  no  statistically  significant  differences  between  modalities  for  minefield  errors. 
However,  between  scenario  maps,  there  were  significant  differences  in  minefield  errors,  with 
scenario  maps  “E”  and  “F”  having  20%  and  24%  more  errors,  respectively,  than  the  mean  for  all 
maps,  indicating  there  was  more  than  likely  a  problem  with  those  two  maps  which  caused  more 
minefield  errors.  A  potential  confound  to  the  results  of  minefield  error  is  that  the  moving  map  and 
baseline  modalities  presented  the  participant  with  a  complete  and  accurate  view  of  the  terrain  via 
the  navigation  display,  including  the  location  of  the  minefield,  whereas  the  other  display 
modalities  relied  on  the  participant  to  perceive  and  react  to  the  minefield  markers  as  seen  solely 
from  the  primary  display.  To  be  more  fair  to  the  different  modalities,  all  scenarios  should  have 
included  an  additional,  unmapped  minefield  or  other  additional  hazard.  Such  a  hazard  would  have 
prevented  the  participant  from  navigating  solely  by  watching  the  moving  map  display. 

Compared  to  the  baseline  modality,  the  head-up  icon  and  moving  map  modalities  took  significantly 
less  time  to  complete  (both  were  36%  faster  than  the  baseline).  The  monaural  speech  modality  was 
significantly  faster  also,  taking  24%  less  time  than  the  baseline  modality,  while  the  spatial  speech, 
spatial  tone,  and  tactile  modalities  were  not  significantly  different  from  the  baseline  modality. 

3.3.2  Situation  Awareness 

The  spatial  tone  waypoint  display  modality  resulted  in  40%  lower  SART  scores,  i.e.,  lower  SA, 
than  the  baseline  display  modality.  No  modality  exhibited  significantly  higher  SART  scores  than 
the  baseline  display.  Among  the  additional  displays,  the  moving  map  waypoint  display  modality 
had  significantly  higher  SART  scores  than  the  monaural  speech,  spatial  speech,  spatial  tone,  and 
tactile  displays.  The  SART  subscale  analysis  showed  that  although  the  supply  subscale  was 
relatively  constant  between  display  modalities,  the  demand  and  understanding  subscales  were 
significantly  different  between  modalities. 

3.3.3  Mental  Workload 

In  experiment  1,  both  the  head-up  icon  and  moving  map  modalities  resulted  in  significantly  lower 
NASA-TLX  scores  (i.e.,  lower  mental  workload  demand)  than  the  baseline  display  modality  by 
57%  and  56%,  respectively.  No  modality  exhibited  significantly  higher  NASA-TLX  scores  than 
the  baseline  display.  However,  the  spatial  tone  modality  had  significantly  higher  workload  scores 
than  the  head-up  icon,  moving  map,  monaural  speech,  spatial  speech,  and  tactile  display 
modalities.  The  NASA-TLX  subscale  analysis  showed  that  the  mental  demand,  temporal  demand, 
performance,  effort,  and  frustration  subscales  were  significantly  different  between  modalities. 

The  only  subscale  that  showed  significant  differences  was  the  performance  subscale. 

Although  the  NASA-TLX  is  not  intended  to  measure  absolute  mental  workload,  it  is  interesting 
to  note  that  the  overall  mean  NASA-TLX  score  across  all  modalities  is  approximately  33  on  a 
total  scale  of  100.  This  could  imply  that  the  designed  navigation  task,  in  general,  places  a  low 
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mental  demand  on  the  partieipant,  eontrary  to  the  IMPRINT  analyses  by  Mitehell  et  al.  (2004). 
Further  studies  should  be  eondueted  with  an  increased  mental  workload,  possibly  including 
multiple  simultaneous  tasks. 

3.3.4  Modality  Preference 

From  the  ANOVA  of  the  modality  evaluation  ratings,  it  is  evident  that  the  moving  map  modality 
was  preferred  to  the  baseline,  spatial  speech,  spatial  tone,  and  tactile  modalities  and  that  the  head- 
up  icon  modality  was  preferred  to  the  baseline  and  spatial  tone  modalities.  The  spatial  tone 
modality  had  the  lowest  evaluation  rating  of  all  the  modalities.  The  high  internal  consistency,  as 
measured  by  Chronbach’s  alpha,  is  a  good  indication  that  the  seven  question  items  of  the  designed 
questionnaire  were  measuring  the  same  dimension. 

3.3.5  Summary 

From  the  post  hoc  analyses  of  each  dependent  variable,  it  is  clear  that  the  head-up  icon  and 
moving  map  were  the  two  consistently  best  display  modalities.  These  two  display  modalities 
presented  information  visually  in  a  way  that  is  more  intuitive  with  much  less  mental  calculation 
than  the  baseline  modality.  In  agreement  with  MRT,  the  visually  oriented  task  of  navigation, 
using  a  visual  display  that  enables  ease  of  understanding  information,  allows  for  improved 
performance. 

Across  all  the  dependent  variables,  there  were  no  significant  differences  between  the  monaural 
and  spatial  speech  modalities.  However,  no  additional  auditory  tasks  were  included  in  the 
experiment.  It  could  be  that  at  the  low  level  of  auditory  demand  for  this  navigation  task,  there  is 
not  the  anticipated  improvement  that  spatial  audio  can  afford. 

The  spatial  tone  modality  showed  poor  results  across  all  dependent  variables  in  experiment  1 , 
most  apparently  in  the  modality  evaluation  and  NASA- TLX  results.  Examining  the  NASA- TLX 
subscales  revealed  that  the  frustration  subscale  score  is  markedly  higher  than  all  other  modalities 
for  the  spatial  tone  display  modality.  The  spatial  tone  modality  received  much  higher  scores  for 
the  “annoying”  question  item  of  the  modality  evaluation  questionnaire.  It  is  believed  that  the 
extreme  annoyance  of  the  spatial  tone  could  have  caused  the  participants  to  rate  scores  lower 
across  the  other  dependent  variables. 
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4.  Experiment  2 


4.1  Method 

4.1.1  Participants 

The  participants  for  the  second  experiment  were  18  male  Soldiers,  ranging  in  age  from  18  to  29 
years  (M=  22.3  years,  SD  =  3.32  years).  Participants  had  from  0.5  to  9.5  years  of  military  service 
(M=  3.65  years,  SD  =  2.61  years).  Participants  were  from  a  diverse  range  of  MOSs.  All  the 
participants  had  normal  visual  acuity  and  color  vision  in  both  eyes.  Participants  were  not  given  a 
hearing  threshold  level  test,  but  all  participants  reported  no  suspected  hearing  loss  or  damage.  The 
voluntary,  fully  informed  consent  of  the  persons  used  in  this  research  was  obtained  as  required  by 
32  CFR  219  and  AR  70-25.  The  investigator  has  adhered  to  the  policies  for  the  protection  of 
human  subjects,  as  prescribed  in  AR  70-25. 

4.1.2  Apparatus 

As  in  the  first  experiment,  participants  sat  at  a  computer  workstation  with  two  PCs  equipped 
identically  to  those  in  the  first  experiment  except  that  a  Sennheiser  HD  280  Pro  circumaural  stereo 
headset  was  used  to  provide  audio  from  the  PC.  VBSl  was  again  used  as  the  virtual  environment 
for  experiment  2. 

4.1.3  Task 

Similar  to  the  first  experiment,  each  navigation  scenario  consisted  of  a  series  of  three  waypoints, 
each  including  a  visible  object.  The  navigation  mission  consisted  of  three  segments,  each  a  2-km 
distance.  Two  of  the  three  navigation  segments  included  identifying  and  navigating  around  a  200-m 
by  1000-m  minefield  marked  with  danger  signs  around  the  perimeter.  One  of  the  minefields  was 
identified  on  the  navigation  map  (in  the  baseline  and  moving  map  display  modalities),  and  the  other 
minefield  was  intentionally  not  identified  on  the  navigation  map  to  force  the  participant  to  pay 
attention  to  the  primary  display  and  not  solely  rely  on  the  navigation  display.  Diagrams  of  the  six 
scenario  maps  are  included  in  appendix  E.  Scripting  of  the  scenarios  in  VBSl  was  greatly  improved 
for  this  experiment. 

4.1.4  Questionnaires 

The  same  questionnaires  as  the  first  experiment  were  used:  the  SART,  NASA- TLX,  and  modality 
evaluation  questionnaires. 
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4,1,5  Experimental  Design 

Experiment  2  was  also  a  single  factor  within-subjects  design.  The  independent  and  dependent 
variables  were  the  same  as  for  the  first  experiment;  waypoint  display  modality  was  again  the 
independent  variable,  while  navigation  performance,  SA,  mental  workload,  and  modality  prefer¬ 
ence  remained  the  dependent  variables.  The  same  measures  were  used  for  each  dependent  variable 
as  in  the  first  experiment. 

For  experiment  2,  there  were  six  levels  of  the  waypoint  display  modality  independent  variable: 
baseline,  head-up  icon,  moving  map,  monaural  speech,  spatial  speech,  and  tactile.  The  spatial 
audio  tone  modality  was  removed  for  this  experiment  because  of  the  poor  results  of  the  modality 
in  the  first  experiment.  The  baseline,  head-up  icon,  and  moving  map  conditions  were  identical  as 
those  used  in  experiment  1 .  The  only  difference  in  the  monaural  speech,  spatial  speech,  and  tactile 
conditions  was  that  direction  cues  were  given  every  10  seconds  rather  than  every  100  meters 
traveled.  Each  condition  is  now  described  in  more  detail. 

•  Baseline  -  A  static,  2-D  overhead  map  provided  on  the  navigation  display  showed  the 
starting  position,  waypoint  locations,  GPS  grid  lines,  terrain  features,  and  landmarks  but 
did  not  update  or  show  the  location  or  heading  of  the  vehicle  (see  figure  5).  The  participant 
used  the  primary  display  along  with  the  GPS  display  (in  standard  EITM  format)  and  compass 
to  determine  his  current  location  and  heading  to  the  next  waypoint.  Close  views  of  the  GPS 
display  and  compass  display  areas  are  presented  in  figures  6  and  7,  respectively. 

•  Head-up  icon  -  A  yellow,  square  icon  was  overlaid  on  the  primary  display  indicating  the 
location  of  the  waypoint  (see  figure  8).  When  the  waypoint  was  out  of  the  driver’s  FOV, 
the  icon  changed  to  a  triangular  arrow,  indicating  the  direction  toward  the  waypoint.  The 
word  “waypoint”  and  the  distance  to  the  waypoint  were  shown  next  to  the  icon. 

•  Moving  map  -  A  2-D  overhead  map  provided  on  the  navigation  display  showed  the 
starting  position,  waypoint  locations,  GPS  coordinates,  terrain  features,  and  landmarks, 
exactly  the  same  as  the  baseline  condition,  but  also  showed  the  location  of  the  vehicle 
updated  in  real  time  (see  figure  9).  The  vehicle  icon  was  at  the  center  of  the  map  and 
rotated  to  show  heading,  while  the  map  translated  up/down/left/right  to  show  position. 

•  Monaural  speech  -  Every  10  seconds,  a  synthetic  speech  voice  (generated  with  the  AT&T 
Natural  Voices  Text-to-Speech  Engine)  announced  the  relative  direction  to  the  next 
waypoint  (e.g.,  “waypoint  at  10  o’clock”).  Distance  to  the  next  waypoint  was  communicated 
by  the  same  synthetic  speech  voice,  at  designated  distance  intervals  of  1000,  500,  400,  300, 
200,  and  100  meters  from  the  waypoint  (e.g.,  “400  meters  to  waypoint”).  The  heading  and 
distance  information  was  presented  at  equal  loudness  to  both  ears. 

•  Spatial  speech  -  Emersys  Maven  3D  Professional  was  used  to  create  spatial  audio  sound 
representa-tions  of  the  same  synthetic  speech  voice  used  in  the  monaural  speech  condition. 
The  software  uses  a  generic  head-related  transform  function  to  produce  a  stereo  sound 
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representation  of  monaural  sounds.  Every  10  seconds,  the  synthetic  speech  voice  announced 
the  relative  direction  to  the  next  waypoint  which  appeared  to  emanate  from  the  direction  of 
the  next  waypoint  (e.g.,  “waypoint  at  1  o’clock”  is  heard  from  the  30-degree  azimuth). 
Distance  was  presented  in  the  same  manner  as  the  monaural  speech  condition. 

•  Tactile  -  The  MIT  WTCU  (see  figure  10)  was  used  with  12  vibro-tactile  motors  positioned 
at  the  12  clock  positions  in  a  belt  worn  around  the  participant’s  abdomen.  Every  10  seconds, 
the  WTCU  provided  two  1000-ms  bursts  with  a  333-ms  rest  in  the  relative  direction  to  the 
next  waypoint  (e.g.,  if  the  next  waypoint  was  at  3  o’clock,  the  stimulus  vibrated  at  the 
participant’s  right  side).  The  vibro-tactile  motors  used  vibrated  at  approximately  70  Hz. 
Distance  was  presented  in  the  same  manner  as  the  monaural  speech  modality. 

The  navigation  display  was  only  provided  for  the  baseline  and  moving  map  conditions.  Eor  all 
the  conditions,  when  a  waypoint  was  reached,  the  synthetic  speech  voice  announced  “waypoint 
reached”.  All  the  generated  sound  files  (synthetic  speech  and  spatial  synthetic  speech)  were 
normalized  to  98%  of  maximum  volume  with  Adobe  Audition. 

We  created  a  Greco-Eatin  square  design  by  superimposing  two  balanced  Eatin  squares,  one  for  the 
display  modality  condition,  and  one  for  the  navigation  map  (see  table  11).  The  number  and  letter 
combination  in  each  cell  of  table  1 1  indicates  the  scenario  that  was  used  for  each  participant;  the 
display  modality  (1-6)  is  shown  first,  and  the  scenario  map  (A-E)  is  shown  second.  Because  of  the 
nature  of  the  6x6  Greco-Eatin  square,  it  was  impossible  to  counterbalance  both  presentation  order 
and  scenario  map,  so  a  design  was  chosen  that  balanced  modality  and  scenario  map,  sacrificing  the 
balancing  of  scenario  map  and  presentation  order. 


Table  11.  Greco-Latin  square  design  for  experiment  2. 


Order 

1st 

2nd 

3rd 

4th 

5th 

6th 

Participant 

Number 

1,  7, 13 

lA 

2B 

6C 

3D 

5E 

4F 

2,  8, 14 

2A 

3B 

1C 

4D 

6E 

5F 

3,  9, 15 

3A 

4B 

2C 

5D 

IE 

6F 

4, 10, 16 

4A 

5B 

3C 

6D 

2E 

IF 

5, 11, 17 

5A 

6B 

4C 

ID 

3E 

2F 

6, 12, 18 

6A 

IB 

5C 

2D 

4E 

3F 

4,1.6  Procedure 

Volunteers  received  an  overview  of  the  experiment  and  details  of  the  procedures  and  were 
informed  of  any  risks  involved  in  their  participation.  The  volunteers  read  aloud  and  were  asked  to 
sign  an  informed  consent  form  if  they  agreed  to  participate.  After  their  fully  informed,  voluntary 
consent  was  obtained,  participants  completed  a  demographics  questionnaire,  visual  acuity  test,  and 
a  color  deficiency  test. 

Each  participant  received  2  hours’  training  and  practice  with  VBSl.  Eirst,  participants  were 
familiarized  with  VBSl,  focusing  on  the  controls  via  the  keyboard  and  mouse.  Participants  then 
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completed  praetiee  scenarios  for  eaeh  of  the  six  experimental  conditions,  each  lasting  approximately 
10  minutes.  Partieipants  also  received  instruetions  and  praetieed  eompleting  the  NASA-TLX, 
SART,  and  modality  evaluation  questionnaires  after  the  final  praetiee  seenario. 

After  training  was  eomplete,  partieipants  eompleted  the  six  experimental  eonditions.  Before  eaeh 
seenario,  the  partieipants  received  a  short  briefing  reminding  them  of  the  visible  markers  at  the 
waypoints;  they  were  told  to  be  aware  of  and  navigate  around  the  minefields  and  were  told  to 
reach  each  of  the  waypoints  as  quiekly  as  possible.  After  eompleting  eaeh  experimental  eondition, 
the  partieipant  eompleted  the  NASA-TLX,  SART,  and  modality  evaluation  questionnaires. 

4,2  Results 

SPSS  for  Windows,  Release  13,  was  again  used  for  statistical  analysis.  Each  dependent  variable 
was  analyzed  for  signifieant  differenees  between  waypoint  display  modalities  with  a  repeated 
measures  ANOVA.  Additionally,  a  separate  repeated  measures  ANOVA  was  eondueted  with 
seenario  map  and  order  as  the  independent  variable.  Order  and  seenario  map  eould  not  be  tested 
individually  sinee  seenario  map  and  order  were  the  same  in  each  condition.  When  signifieant 
differenees  were  present  (a  <  .05),  post  hoc  tests  were  performed  with  the  LSD  method.  There 
were  signifieant  effeets  of  waypoint  display  modality  for  all  but  one  dependent  variable  and  no 
signifieant  effeets  of  scenario  map  and  order  on  any  of  the  dependent  variables.  A  summary  of 
the  repeated  measures  ANOVA  results  for  modality  and  seenario  map  and  order  is  presented  in 
tables  12  and  13,  respeetively. 


Table  12.  Repeated  measures  ANOVA  summary,  waypoint  display  modality 
as  independent  variable. 


Dependent  Variable 

F  (5, 85) 

Sig. 

Minefield  Navigation  Error 

0.722 

p  =  .609 

Navigation  Time 

12.304 

p  <  .001 

Situation  Awareness  (SART) 

2.520 

p  =  .035 

Mental  Workload  (NASA-TLX) 

8.239 

p  <  .001 

Modality  Rating 

7.291 

p  <  .001 

Table  13.  Repeated  measures  ANOVA  summary,  scenario  map  and  order  as 
independent  variable. 


Dependent  Variable 

F  (5,  85) 

Sig. 

Minefield  Navigation  Error 

1.811 

p  =  .119 

Navigation  Time 

0.991 

p  =  .428 

Situation  Awareness  (SART) 

1.262 

p  =  .288 

Mental  Workload  (NASA-TLX) 

0.930 

p  =  .466 

Modality  Preference 

0.929 

p  =  .467 

4.2,1  Navigation  Performance 

Two  metrics  were  again  used  for  navigation  performanee:  minefield  navigation  errors  and  navi¬ 
gation  time.  In  this  experiment,  there  were  two  types  of  minefields:  mapped  and  unmapped. 
This  distinction  is  only  relevant  to  the  baseline  and  moving  map  modalities  since  those  were  the 
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only  modalities  that  provided  the  navigation  display  (for  the  other  modalities,  all  the  minefields 
are  essentially  “unmapped”  since  there  is  no  navigation  display  to  show  a  map). 

The  percentage  of  minefield  navigation  errors  versus  waypoint  display  modality  is  presented  in 
figure  19,  with  stacked  bars  representing  mapped  and  unmapped  minefield  navigation  errors. 
From  the  repeated  measures  ANOVA,  no  significant  differences  were  found  between  waypoint 
display  modalities  for  minefield  navigation  errors. 


Baseline  Head  Up  Moving  Mono  Spatial  Tactile 

Icon  Map  Speech  Speech 

Modality 

Figure  19.  Minefield  navigation  errors  versus  waypoint  display  modality. 

Although  no  significant  differences  were  identified  between  scenario  map/order  for  minefield 
navigation  errors,  the  significance  level  for  scenario  map/order  was  much  greater  than  for 
waypoint  display  modality  {p  =  .\\9,p=  .609,  respectively).  Figure  20  presents  the  percentage 
of  minefield  navigation  errors  versus  scenario  map. 

Unlike  experiment  1,  there  were  no  missing  values  to  be  computed.  Two  extreme  data  outliers 
were  identified  for  navigation  time;  however,  after  different  data  outlier  strategies  were  weighted, 
the  outliers  were  left  unchanged  in  the  data.  Mean  navigation  time  versus  waypoint  display 
modality  is  presented  in  figure  21 .  Significant  differences  existed  between  waypoint  display 
modalities  for  navigation  time,  F  (5,  85)  =  12.304,/?  <  .001.  Post  hoc  analysis  indicated  that  total 
navigation  time  was  greater  (i.e.,  slower)  in  the  baseline  modality  than  all  other  waypoint  display 
modalities  (all  p  <  .01).  Navigation  time  in  the  moving  map  modality  was  less  (i.e.,  faster)  than 
the  baseline,  monaural  speech,  spatial  speech,  and  tactile  waypoint  display  modalities  (all 
p  <  .01).  Navigation  time  in  the  head-up  icon  modality  was  less  (i.e.,  faster)  than  the  baseline, 
spatial  speech,  and  tactile  waypoint  display  modalities  (all  p  <  .01).  Significant  differences  did  not 
exist  between  the  head-up  icon  and  moving  map,  or  the  monaural  speech,  spatial  speech,  and 
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tactile  waypoint  display  modalities  (all  p  >  .05).  A  summary  of  navigation  time  mean  differenees 
is  presented  in  table  14. 
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Figure  20.  Minefield  navigation  errors  versus  waypoint  display  modality. 
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Figure  21.  Mean  navigation  time  versus  waypoint  display  modality.  (Error  bars 
show  95%  confidence  interval.) 
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Table  14.  Mean  differences  between  modalities  in  navigation  time. 


Modality 

Head-np 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Tactile 

Baseline 

313.838 

{p  =  .002) 

326.889 

{p  =  .001) 

297.056 

(p  =  .002) 

272.944 
(p  =  .005) 

277.278 

(p  =  .006) 

Head-Up  Icon 

13.056 

(p=.233) 

-16.778 

(p  =  .082) 

-40.889 

(p  =  .010) 

-36.556 
(p  =  .007) 

Moving  Map 

-29.833 

(p  =  .002) 

-53.944 

(p  <  .001) 

-49.611 

(p  <  .001) 

Monanral 

Speech 

-24.111 
{p  =  .056 

-19.778 

(P-.151) 

Spatial  Speech 

4.333 

(p=J65) 

Bold  blocks  indicate  significant  differences. 


4,2,2  Situation  Awareness 

Mean  overall  SART  score  versus  waypoint  display  modality  is  presented  in  figure  22.  Marginally 
significant  differences  were  found  between  waypoint  display  modalities  for  overall  SART  score, 

F  (5,  85)  =  2.520, />  =  .035.  Post  hoc  analysis  indicated  that  overall  SART  scores  were  lower 
(indicating  a  lower  level  of  SA)  in  the  tactile  modality  than  the  baseline,  moving  map,  and  spatial 
speech  waypoint  display  modalities  (all  p  <  .05).  Significant  differences  did  not  exist  between  the 
baseline,  head-up  icon,  moving  map,  monaural  speech,  or  spatial  speech  waypoint  display 
modalities  (all p  >  .05).  A  summary  of  SART  score  mean  differences  is  presented  in  table  15. 
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Figure  22.  Overall  SART  score  versus  waypoint  display  modality.  (Error 
bars  show  95%  confidence  interval.) 
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Table  15.  Mean  differences  between  modalities  in  SART  score. 


Modality 

Head-np 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Tactile 

Baseline 

-2.833 
(p  =  .374) 

-2.833 

(p=.393) 

3.278 

(P=.177) 

0.500 
{p  =  .803) 

5.556 
{p  =  .033) 

Head-  Up  Icon 

0.000 

6.111 

(p=.153) 

3.333 

(p=.296) 

8.389 

(p  =  .060) 

Moving  Map 

6.111 

(p=.106) 

3.333 

{p  =  .288) 

8.389 
(p  =  .048) 

Monanral 

Speech 

-2.778 
{p  =  .232) 

2.278 

(P=.157) 

Spatial  Speech 

5.056 
(p  =  .047) 

Bold  blocks  indicate  significant  differences. 


Figure  23  illustrates  the  three  SART  subseale  seores  (demand,  supply,  and  understanding)  versus 
display  modality.  The  repeated  measures  ANOVA  revealed  signifieant  differenees  only  in  the 
demand  subscale.  The  supply  and  understanding  subscales  did  not  reveal  any  significant  differ¬ 
ences  for  display  modality.  The  results  of  the  ANOVA  are  summarized  in  table  16.  Results  of 
post  hoc  analyses  of  SART  subscale  scores  with  significant  differences  are  included  in  appendix  F. 
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Figure  23.  SART  subscale  scores  versus  waypoint  display  modality.  (Error 
bars  show  95%  confidence  interval.) 


Table  16.  Repeated  measures  ANOVA  summary  of  SART  subscale  scores. 


SART  Snhscale 

F  (5,  85) 

Sig. 

Demand 

5.649 

p<.001 

Supply 

2.294 

p  =  .052 

Understanding 

2.122 

p  =  .070 
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4,2,3  Mental  Workload 


Mean  overall  NASA-TLX  score  versus  waypoint  display  modality  is  presented  in  figure  24. 
Significant  differences  were  found  between  waypoint  display  modalities  for  overall  NASA-TLX 
score,  F  (5,  85)  =  8.239,/?  <  .001.  Post  hoc  analysis  indicated  that  overall  NASA-TLX  scores 
were  greater  (indicating  higher  workload)  in  the  baseline  modality  than  all  other  waypoint  display 
modalities  (all  p  <  .05).  Overall  NASA-TLX  scores  were  lower  (indicating  lower  workload)  in 
the  head-up  icon  modality  than  the  baseline,  spatial  speech,  and  tactile  waypoint  display 
modalities  (all  p  <  .05).  Significant  differences  did  not  exist  between  the  head-up  icon,  moving 
map,  or  monaural  speech  modalities  (all  p  >  .05),  nor  did  significant  differences  exist  between  the 
moving  map,  monaural  speech,  spatial  speech,  and  tactile  modalities  (all p  >  .05).  A  summary  of 
NASA-TLX  score  mean  differences  is  presented  in  table  17. 
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Figure  24.  Overall  NASA-TLX  score  versus  waypoint  display  modality. 

(Error  bars  show  95%  confidence  interval.) 
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Table  17.  Mean  differences  between  modalities  in  NASA-TLX  score. 


Modality 

Head-np 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Tactile 

Baseline 

30.389 

(p  =  .001) 

27.889 
(p  =  .003) 

22.056 

(p  =  .001) 

17.333 
(p  =  .017) 

21.667 
(p  =  .003) 

Head-Up  Icon 

-2.500 
(b  =  .513) 

-8.333 

(p=.087) 

-13.056 
(p  =  .007) 

-8.722 
(p  =  .045) 

Moving  Map 

-5.833 
ip  =  .242) 

-10.556 

(p=.087) 

-6.222 

(p=.264) 

Monanral 

Speech 

-4.722 
(p  =  .132) 

-0.389 

(b=.931) 

Spatial  Speech 

4.333 

(p=.258) 

Bold  blocks  indicate  significant  differences. 
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Figure  25  illustrates  the  NASA-TLX  subseale  seores  (mental  demand,  temporal  demand, 
performanee,  effort,  and  frustration)  versus  display  modality.  An  ANOVA  revealed  signifieant 
differenees  in  the  mental  demand  and  effort  subseales  but  not  the  temporal  demand,  performanee, 
or  frustration  subseales.  The  ANOVA  results  are  summarized  in  table  18.  Post  hoc  analyses  of 
NASA-TLX  subseale  seores  with  signifieant  differenees  are  ineluded  in  appendix  G. 
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Figure  25.  NASA-TLX  subscale  scores  versus  waypoint  display  modality. 


Table  18.  Repeated  measures  ANOVA  summary  of  NASA-TLX  subscale 
scores. 


NASA-TLX  Subscale 

F  (5,  85) 

Sig. 

Mental  Demand 

15.309 

p  <  .001 

Temporal  Demand 

0.185 

p  =  .968 

Performance 

0.899 

p  =  .486 

Effort 

2.881 

p  =  .019 

Frustration 

0.590 

p  =  .708 

4,2,4  Modality  Preference 

Mean  overall  partieipant  evaluation  rating  versus  waypoint  display  modality  for  the  modality 
evaluation  questionnaire  is  presented  in  figure  26.  The  repeated  measures  ANOVA  revealed 
signifieant  differences  between  waypoint  display  modalities  for  evaluation  rating,  F  (5,  85)  = 
7.291,;?  <  .001.  Post  hoc  analysis  indicated  that  evaluation  ratings  were  lower  in  the  baseline 
modality  than  all  other  waypoint  display  modalities  (all p  <  .05).  Evaluation  ratings  were  higher 
for  the  head-up  icon  modality  than  the  baseline,  monaural  speech,  and  tactile  waypoint  display 
modalities  (all  p  <  .05).  Significant  differences  did  not  exist  between  the  head-up  icon,  moving 
map,  and  spatial  speech  display  modalities,  or  the  moving  map,  monaural  speech,  spatial  speech. 
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and  tactile  waypoint  display  modalities  (all  p  >  .05).  A  summary  of  evaluation  rating  mean 
differenees  is  presented  in  table  19. 

To  further  analyze  the  modality  evaluation  questionnaire,  inter-item  eorrelations  (table  20)  and 
Chronbach’s  alpha  were  computed.  Chronbach’s  alpha  was  calculated  to  be  0.859  for  the  scale. 
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Figure  26.  Participant  evaluation  rating  versus  waypoint  display  modality. 
(Error  bars  show  95%  confidence  interval.) 


Table  19.  Mean  differences  between  modalities  in  evaluation  rating. 


Modality 

Head-np 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Tactile 

Baseline 

-1.048 

(p  =  .001) 

-0.762 
(p  =  .014) 

-0.690 
(p  =  .009) 

-0.754 
(p  =  .004) 

-0.619 
(p  =  .014) 

Head-Up  Icon 

0.286 
(p  =  .082) 

0.357 
(P  =  .012) 

0.294 
{p  =  .098) 

0.426 
{p  =  .009) 

Moving  Map 

0.071 

(P--572) 

0.008 
{p  =  .948) 

0.143 

(p=-457) 

Monanral 

Speech 

-0.063 
{p  =  .505) 

0.071 

(P=.519) 

Spatial  Speech 

0.135 

(p=.387) 

Bold  blocks  indicate  significant  differences. 
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Table  20.  Inter-item  correlations  for  the  modality  evaluation  questionnaire. 


Question  Item 

Efficient 

Simple 

Noticed 

Attention 

Helpful 

Effective 

.811 

.811 

.582 

.627 

.387 

Efficient 

.766 

.621 

.729 

.344 

Simple 

.629 

.714 

.408 

Noticed 

.839 

.423 

Attention 

.483 

All  correlations  significant  at  the  0.01  level  (2-tailed). 


4,3  Discussion 

The  hypothesis  that  the  alternate  display  modalities  would  provide  greater  navigation  performance, 
increased  SA,  and  decreased  mental  workload  than  the  baseline  display  configuration  is  not  fully 
supported  by  the  results.  There  were  no  significant  differences  between  the  baseline  and  several  of 
the  alternate  modalities  for  the  SA  dependent  variables  or  an  alternate  modality  (tactile)  that 
resulted  in  lower  SA  than  the  baseline  modality,  which  directly  refutes  the  hypothesis. 

The  results  of  experiment  2  support  the  hypothesis  that  there  would  be  significant  differences 
between  waypoint  display  modalities.  Except  for  minefield  navigation  errors,  all  the  dependent 
variables  exhibited  significant  differences  between  display  modalities. 

4,3,1  Navigation  Performance 

There  were  no  statistically  significant  differences  between  modalities  for  minefield  errors. 

Results  of  the  repeated  measures  ANOVA  did  not  show  that  minefield  errors  were  affected  by 
scenario  map  and  order,  but  the  significance  level  was  much  greater  for  scenario  map  and  order 
than  for  waypoint  display  modality  {p=  .\\9,p  =  .609,  respectively),  indicating  there  was  more 
likely  an  effect  because  of  scenario  map/order  than  display  modality. 

To  address  the  minefield  navigation  error  problem  identified  in  experiment  1,  the  navigation 
scenarios  in  experiment  2  included  both  a  mapped  and  unmapped  minefield  (this  distinction  is 
only  relevant  to  the  baseline  and  moving  map  modalities).  Examining  the  differences  between 
the  mapped  and  unmapped  minefield  navigation  errors  in  the  baseline  display  modality  showed 
that  there  was  not  a  significant  difference  in  a  paired  sample  T  test  (p  =  .331).  There  were  no 
minefield  navigation  errors  for  the  moving  map  modality,  so  there  was  no  comparison  for  that 
modality. 

Compared  to  the  baseline  modality,  all  the  modalities  took  significantly  less  time  to  complete. 
Eastest  were  the  head-up  icon  and  moving  map  modalities  which  were  45%  and  47%  faster  than 
baseline,  respectively.  The  monaural  speech,  spatial  speech  and  tactile  modalities  were 
significantly  faster  also,  taking  42%,  39%,  and  40%  less  time  than  the  baseline  modality. 
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4.3.2  Situation  Awareness 

In  experiment  2,  there  were  marginally  significant  differences  in  overall  SART  scores  between 
waypoint  modalities.  Post  hoc  comparison  revealed  differences  between  only  the  tactile  and 
baseline,  moving  map,  and  spatial  speech  modalities.  The  tactile  display  may  have  generated 
lower  SART  scores  because  of  the  difference  in  information  quantity  given  by  the  visual  displays. 
The  tactile  display,  monaural  and  spatial  speech  auditory  displays  facilitate  a  local  awareness  of 
the  environment,  rather  than  the  global  picture  that  the  visual  displays  enable.  Combining 
displays  into  a  multimodal  interface  would  be  worth  investigation.  The  SART  subscale  analysis 
showed  that  the  demand  subscale  was  significantly  different  between  modalities  while  the  supply 
and  understanding  subscales  were  relatively  constant  between  display  modalities. 

4.3.3  Mental  Workload 

As  in  the  first  experiment,  both  the  head-up  icon  and  moving  map  modalities  resulted  in  signifi¬ 
cantly  lower  NASA-TLX  scores  (i.e.,  lower  mental  workload  demand)  than  the  baseline  display 
modality  by  50%  and  45%,  respectively.  All  modalities  in  experiment  2  resulted  in  significantly 
lower  NASA-TLX  scores  than  the  baseline:  monaural  speech  by  36%,  spatial  speech  by  28%,  and 
tactile  by  35%.  The  NASA-TLX  subscale  analysis  showed  that  the  mental  demand,  perform-ance, 
and  effort  subscales  were  significantly  different  between  modalities.  The  mental  demand  subscore 
was  much  higher  in  the  baseline  display  modality  because  of  the  mental  calculation  required  to 
determine  the  appropriate  course  to  the  waypoint  when  no  automated  guidance  is  provided. 

The  overall  mean  NASA-TLX  score  across  all  modalities  for  experiment  2  was  approximately  42 
on  a  total  scale  of  100.  As  mentioned  for  experiment  1,  the  NASA-TLX  is  not  intended  to  measure 
absolute  mental  workload;  however,  this  could  imply  that  the  navigation  task  in  this  experiment 
(which  was  nearly  identical  to  the  task  in  experiment  1)  places  a  low  mental  demand  on  the  partici¬ 
pant.  Again,  further  investigation  into  navigation  tasks  with  an  increased  mental  workload  should 
be  conducted  to  assess  the  effects  of  display  modality  in  a  more  demanding  task. 

4.3.4  Modality  Preference 

From  the  ANOVA  of  the  modality  evaluation  ratings,  it  is  evident  that  the  moving  map  modality 
was  preferred  to  the  baseline  modality  and  that  the  head-up  icon  modality  was  preferred  to  the 
baseline,  monaural  speech,  and  tactile  modalities.  The  baseline  modality  had  the  lowest 
evaluation  rating  of  all  the  modalities.  As  in  experiment  1 ,  the  modality  preference  questionnaire 
exhibited  high  internal  consistency,  as  measured  by  Chronbach’s  alpha.  Again,  this  is  a  good 
indication  that  the  seven  question  items  of  the  designed  questionnaire  were  measuring  the  same 
dimension. 
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4,3,5  Summary 

Similar  to  experiment  1 ,  the  post  hoc  analyses  of  each  dependent  variable  revealed  that  the  head- 
up  icon  and  moving  map  were  the  two  consistently  best  display  modalities  for  experiment  2. 
These  two  display  modalities  presented  information  visually  in  a  way  that  is  more  intuitive  with 
much  less  mental  calculation  than  the  baseline  modality.  In  agreement  with  MRT,  the  visually 
oriented  task  of  navigation,  which  used  a  visual  display  that  enabled  ease  of  understanding 
information,  allows  for  improved  performance. 

In  experiment  2,  just  as  in  the  first  experiment,  there  were  no  significant  differences  between  the 
monaural  and  spatial  speech  modalities  across  all  the  dependent  variables.  As  previously 
mentioned,  no  additional  auditory  tasks  were  included  in  the  experiment.  One  explanation  is  that 
with  the  low  level  of  auditory  demand  for  this  navigation  task,  there  was  not  the  anticipated 
improvement  that  spatial  audio  would  allow. 

Like  the  first  experiment,  the  tactile  display  modality  was  not  significantly  different  than  the 
monaural  speech  or  spatial  speech  modalities  (except  for  marginal  difference  between  the  tactile 
and  spatial  speech  modality  in  overall  SART  scores).  It  would  be  interesting  to  see  if  in  a  higher 
workload  environment,  especially  one  with  high  auditory  demand,  the  tactile  display  would 
result  in  improved  performance. 


5.  Conclusions 


The  results  of  these  experiments  indicate  that  the  standard  method  of  navigation,  represented  by 
the  baseline  display  modality,  which  provides  a  static  map  and  forces  the  driver  to  determine  the 
appropriate  course  to  take,  is  time  consuming  and  mentally  demanding.  The  augmented  visual 
displays,  represented  by  the  head-up  icon  and  moving  map  displays,  significantly  reduced 
navigation  time,  maintained  SA,  and  drastically  reduced  mental  workload  over  the  baseline 
display. 

The  effect  of  channel  off  loading,  changing  the  display  modality  from  visual  to  auditory  or  tactile, 
has  not  resulted  in  improved  performance  in  these  experiments,  perhaps  because  of  the  low  task 
demand  presented.  Future  research  should  increase  the  overall  task  demand  by  increasing  the 
visual  and  auditory  demands  significantly.  Additional  tasks  including  visual  search  for  targets 
and  communications  monitoring,  which  are  significant  real-life  factors  increasing  the  mental 
demands  of  tactical  vehicle  drivers,  should  be  considered. 
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Appendix  A.  Scenario  Maps  (Experiment  1) 


Figure  A- 1.  Map  A. 


Figure  A-2.  Map  B. 
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Figure  A- 3.  Map  C. 
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Figure  A- 5.  Map  E. 


Figure  A- 6.  Map  F. 
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Figure  A- 7.  Map  G. 
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Appendix  B.  Questionnaires 


Situational  Awareness  Rating  Technique  (SART) 


Please  enter  your  participant  ID; 


Please  enter  the  mission  ID: 


Instructions:  Please  answer  these  questions  with  regard  to  the  preceding  scenario- 


instability  of  Situation: 

How  changeable  Is  the  situation?  Is  the  situation  highly  unstable  and  likely  to  change  suddenly  (high),  or  is  it  very  stable  and  straight  forward 
(low)? 

^Low  r  2  <"3  ^4  r  5  C  e  ^  7  ("8  ('9  High 

Complexity  of  Situation: 

How  complicated  is  the  situation?  Is  it  complex  with  many  interrelated  components  (high)  or  is  it  simple  and  straightforward  (low)? 

(~Low  ("l  C  2  ("4  ^5  ^7  <"8  <'9  <"  High 

Variability  of  Situation: 

How  many  variables  are  changing  in  the  situation?  Are  there  are  large  number  of  factors  varying  (high)  or  are  there  very  few  variables 
changing  (low)? 

^Low  ('2  <“3  ^4  Ts  re  C  7  r  Q  rg  r  High 

Arousal: 

How  aroused  are  you  in  the  situation?  Are  you  alert  and  ready  for  activity  (high)  or  do  you  have  a  low  degree  of  alertness  (low)? 

Low  ^2  ^3  f~4  ^5  ^7  ^8  ^9  High 

Concentration  of  Attention: 

How  much  are  you  concentrating  on  the  situation?  Are  you  bringing  all  your  thoughts  to  bear  (high)  or  is  your  attention  elsewhere  (low)? 

Low  Cl  c  2  c  2  r  4  r  s  C  6  C  7  C  8  Tg  C  High 

Division  of  Attention: 

How  much  is  your  attention  divided  in  the  situation?  Are  you  concentrating  on  many  aspects  of  the  situation  (high)  or  focussed  on  only  one 
(low)? 

^~Low  C  2  C  3  r  4  C  s  C  e  C  7  r  8  C  g  C  High 

Spare  Mental  Capacity: 

How  much  mental  capacity  do  you  have  to  spare  in  the  situation?  Do  you  have  sufficient  capacity  to  attend  to  many  variables  (high)  or 
nothing  to  spare  at  all  (low)? 

C  Low  ^1  C  2  C  3  C  4  C  s  C  7  C  8  C  g  C  High 

Information  Quantity: 

How  much  information  have  you  gained  about  the  situation?  Have  you  received  and  understood  a  great  deal  of  knowledge  (high)  or  very  little 
(low)? 

C  Low  C  2  C  3  C  4  C  s  C  8  C  7  ("g  C  g  C  High 

Information  Quality: 

How  good  is  the  information  you  have  gained  about  the  situation?  Is  the  knowledge  communicated  very  useful  (high)  or  is  it  a  new  situation 
(low)? 

C  Low  C  2  C  3  C  4  C  s  C  8  C  7  C  8  C  g  C  High 

Familiarity  with  Situation: 

How  familiar  are  you  with  the  situation?  Do  you  have  a  great  deal  of  relevant  experience  (high)  or  is  it  a  new  situation  (low)? 
c  Low  Cl  r  2  c  3  C  4  r  s  C  8  C  7  r  8  ^9  C  High 

Submit  I 
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NASA  -  Task  Load  Index  (TLX) 


Please  enter  your  participant  ID: 

I - 

Please  enter  the  mission  ID; 


Instructions  for  Part  1:  Please  answer  these  questions  with  regard  to  the  preceding  scenario. 


Mental  Demand: 

How  much  mental  and  perceptual  activity  was  required  (e.g.,  thinking,  deciding,  calculating,  remembering,  looking,  searching,  etc.}? 

<"Low  Cl  f'S  C  ^  f'S  <'6  f'  7  f"8  f"9  C  High 

Temporal  Demand: 

How  much  lime  pressure  did  you  feel  due  to  the  rate  or  pace  at  which  the  tasks  or  task  elements  occurred? 

^'Low  ('l  C  2  C  3  ('a  C  5  6  C  7  f's  f'g  ^  High 

Performance! 

How  successful  do  you  think  you  were  in  accomplishing  the  goals  of  the  task  set  by  the  experimenter  (or  yourself)?  How  satisfied  were  you 
with  your  performance  in  accomplishing  these  goals? 

Good  Cl  C  2  C  3  r  4  r  5  r  6  C  7  r  8  C  9  r  Poor 

Effort: 

How  hard  did  you  have  to  work  (mentally  and  physically)  to  accomplish  your  level  of  performance? 

f“Low  Cl  r  2  c  3  r  4  r  5  r  e  c  7  r  a  c  g  r  High 

Frustration: 

How  insecure,  discouraged,  irritated,  stressed  and  annoyed  versus  secure,  gratified,  content,  relaxed  and  complacent  did  you  feel  during  the 
task? 

<~Low  C  2  C  3  C  4  C  5  C  6  C  7  C  8  C  g  C  High 


Instructions  for  Part  2:  From  the  following  pairs,  select  the  scale  title  that  represents  the  more  important  contributor  to  workload  for  the 
task  you  performed  in  the  proceeding  scenario. 


C  Temporal  Demand 
C  Mental  Demand 

C  Mental  Demand 
C  Performance 

C  Temporal  Demand 
C  Effort 

C  Frustration 
C  Mental  Demand 

C  Effort 
C  Performance 

C  Performance 
C  Frustration 

C  Frustration 
C  Effort 

C  Temporal  Demand 
C  Frustration 

C  Performance 
C  Temporal  Demand 

C  Mental  Demand 
C  Effort 

Submit  I 
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Modality  Evaluation 


Ploasa  enter  your  participant  ZD: 

I - 

Pleaga  enter  the  miasion  ZD: 


Znstructtons:  Please  answer  these  quesQons  with  regard  to  the  preceding  scenario: 


Navigation  was  effective. 

^  Strongly  Agree 
^  Agree 
^  Neutfal 
^  Disagree 
^  Strongly  Disagree 

Navigation  was  efficient. 

^  Strongly  Agree 
Agree 
^  Neutral 
^  Disagree 
^  Strongly  Disagree 

Navigation  was  simple. 

^  Strongly  Agree 
Agree 
^  Neutral 
^  Disagree 
^  Strongly  Disagree 

The  alert  was  easily  noticed. 

^  Strongly  Agree 
^  Agree 
^  Neutral 
^  Disagree 
^  Strongly  Disagree 

The  alert  was  effective  at  getting  my  attention. 
^  Strongly  Agree 
^  Agree 
^  Neutral 
^  Disagree 
^  Strongly  Disagree 

The  alert  was  helpful. 

^  Strongly  Agree 
^  Agree 
I*  Neutral 
^  Disagree 

Strongly  Disagree 

The  alert  was  annoying  or  unnecesary. 

^  Strongly  Agree 
^  Agree 
^  Neutral 
^  Disagree 
^  Strongly  Disagree 

Submit  I 
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Appendix  C.  SART  Subscale  Score  Mean  Differences  (ANOVA post  hoc 
analyses)  (Experiment  1) 


Table  C-1.  Mean  differences  between  modalities  in  SART  demand  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Baseline 

3.000 

{p  =  .080) 

2.643 
(P  =  .018) 

0.000 

-0.500 

(p=.837) 

-4.857 
{p  =  .067) 

Head-Up  Icon 

■ 

-0.357 
{p  =  .705) 

-3.000 

(p=.071) 

-3.500 

{p  =  .080) 

-7.857 

(p  =  .006) 

Moving  Map 

-2.643 
(p  =  .032) 

-3.143 

(p=.122) 

-7.500 
(p  =  .005) 

Monanral 

Speech 

-0.500 
(p  =  .786) 

-4.857 
(p  =  .014) 

Spatial  Speech 

-4.357 
(P  =  .015) 

Bold  blocks  indicate  significant  differences. 


Table  C-2.  Mean  differences  between  modalities  in  SART  demand  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Baseline 

-0.714 
ip  =  .547) 

-2.643 

(p=.108) 

0.643 

(P=.611) 

2.357 
{p  =  .070) 

5.643 

(p  =  .006) 

Head-Up  Icon 

g 

-1.929 
(p  =  .lll) 

1.357 

{p  =  .260) 

3.071 
(p  =  .031) 

6.357 

(p  =  .008) 

Moving  Map 

3.286 
(p  =  .032) 

5.000 
(p  =  .014) 

8.286 
(p  =  .004) 

Monanral 

Speech 

1.714 

(p=.258) 

5.000 

(p  =  .026) 

Spatial  Speech 

3.286 

(p  =  .018) 

Bold  blocks  indicate  significant  differences. 
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Appendix  D.  NASA-TLX  Subscale  Score  Mean  Differences  (ANOVA post  hoc 
analyses)  (Experiment  1) 


Table  D-1.  Mean  differences  between  modalities  in  NASA-TLX  mental  demand  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Baseline 

14.500 

(p  =  .002) 

12.071 
(P  =  .001) 

13.214 
(P  =  .001) 

12.214 
(p  =  .014) 

2.214 

(p=.584) 

Head-Up  Icon 

-2.429 

{p  =  .266) 

-1.286 

(p=.481) 

-2.286 
(p  =  .434) 

-12.286 
(p  =  .006) 

Moving  Map 

bs 

O 

0.143 
(p  =  .965) 

-9.857 

(p  =  .008) 

Monanral 

Speech 

-1.000 
(p  =  .668) 

-II.OOO 
(p  =  .003) 

Spatial  Speech 

-10.000 
(p  =  .005) 

Bold  blocks  indicate  significant  differences. 


Table  D-2.  Mean  differences  between  modalities  in  NASA-TLX  temporal  demand  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Baseline 

3.786 
(P  =  .007) 

3.357 
(p  =  .004) 

II  P 

oo 

Ut  Vt 

O 

o 

-0.357 

(p  =  .808) 

0.643 

(p=.501) 

Head-Up  Icon 

-0.429 
{p  =  .396) 

-2.929 

(p  =  .016) 

-4.143 
(p  =  .029) 

-3.143 
(p  =  .035) 

Moving  Map 

-2.500 
(p  =  .019) 

-3.714 

(p  =  .026) 

-2.714 

(p  =  .026) 

Monanral 

Speech 

-1.214 
(p  =  .366) 

-0.214 

(p=.859) 

Spatial  Speech 

1.000 
(p  =  .488) 

Bold  blocks  indicate  significant  differences. 


Table  D-3.  Mean  differences  between  modalities  in  NASA-TLX  effort  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Baseline 

9.143 
(p  =  .004) 

8.643 
(p  =  .007) 

4.143 

(p=.103) 

6.286 
(p  =  .040) 

1.000 
{p  =  .692) 

Head-Up  Icon 

-0.500 
{p  =  .646) 

-5.000 
(p  =  .032) 

-2.857 
{p  =  .069) 

-8.143 

(p  =  .006) 

Moving  Map 

-4.500 
(p  =  .044) 

-2.357 

(P=.187) 

-7.643 

(p  =  .008) 

Monanral 

Speech 

2.143 

{p  =  .228) 

-3.143 

ip-.Ul) 

Spatial  Speech 

-5.286 

(p  =  .021) 

Bold  blocks  indicate  significant  differences. 
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Table  D-4.  Mean  differences  between  modalities  in  NASA-TLX  frustration  subscale  score. 


Modality 

Head-  Up 
Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Spatial 

Tone 

Baseline 

0.071 
{p  =  .885) 

0.143 

(p=.752) 

-1.500 
{p  =  .363) 

-1.714 

(p=.357) 

-16.143 

(p  =  .002) 

Head-Up  Icon 

0.071 
(p=  .336) 

-1.571 

(p=.335) 

-1.786 

(p=.332) 

-16.214 

(p  =  .002) 

Moving  Map 

-1.643 
{p  =  .308) 

-1.857 

(p=.310) 

-16.286 
(p  =  .002) 

Monanral 

Speech 

■ 

■ 

-0.214 
(P  =  .931) 

-14.643 
(p  =  .005) 

Spatial  Speech 

-14.429 

(p  =  .002) 

Bold  blocks  indicate  significant  differences. 
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Appendix  E.  Scenario  Maps  (Experiment  2) 
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Figure  E-3.  Map  C. 


Figure  E-4.  Map  D. 
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Figure  E-5.  Map  E. 
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Appendix  F.  SART  Subscale  Score  Mean  Differences  (ANOVA post  hoc 
analyses)  (Experiment  2) 


Table  F-1.  Mean  differences  between  modalities  in  SART  demand  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Tactile 

Baseline 

7.167 

(p  <  .001) 

6.667 
(P  =  .001) 

4.833 
(p  =  .003) 

5.722 

(p  <  .001) 

II  9" 
O  00 

Ut  'O 

so 

Head-Up  Icon 

-0.500 
(p  =  .719) 

-2.333 

(p=.191) 

-1.444 

(p=.398) 

-3.778 
(p  =  .013) 

Moving  Map 

-1.833 
(p  =  .367) 

-0.944 

(p  =  .602) 

-3.278 

(p  =  .082) 

Monanral 

Speech 

-0.889 
(p  =  .419) 

-1.444 
(p  =  .296) 

Spatial  Speech 

-2.333 
(p  =  .199) 

Bold  blocks  indicate  significant  differences. 
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Appendix  G  NASA-TLX  Subscale  Score  Mean  Differences  (ANOVA post  hoc 
analyses)  (Experiment  2) 


Table  G-1.  Mean  differences  between  modalities  in  NASA-TLX  mental  demand  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Tactile 

Baseline 

20.444 

(p  <  .001) 

17.056 

(p  <  .001) 

14.333 

(p  <  .001) 

12.222 
(p  =  .001) 

15.389 

(p  <  .001) 

Head-Up  Icon 

-3.389 

(p  =  .161) 

-6.III 
(p  =  .004) 

-8.222 
(p  =  .009) 

-5.056 
(p  =  .019) 

Moving  Map 

-2.722 
(p  =  .264) 

-4.833 

(P=.118) 

-1.667 

(p=A57) 

Monanral 

Speech 

■ 

-2.111 
(P  =  .287) 

1.056 

(p=  .601) 

Spatial  Speech 

3.167 

(P=.2l0) 

Bold  blocks  indicate  significant  differences. 


Table  G-2.  Mean  differences  between  modalities  in  NASA-TLX  effort  subscale  score. 


Modality 

Head-Up 

Icon 

Moving 

Map 

Monanral 

Speech 

Spatial 

Speech 

Tactile 

Baseline 

7.6II 

(p  =  .008) 

5.444 
(p  =  .065) 

4.667 
(P  =  .034) 

5.667 
(p  =  .025) 

4.611 

(p=  .106) 

Head-Up  Icon 

-2.167 
(p  =  .030) 

-2.944 

(p=.181) 

-1.944 

(p=.358) 

-3.000 
(p=  .104) 

Moving  Map 

-0.778 

(p  =  .ni) 

0.222 

(P=.913) 

-0.833 
(p  =  .654) 

Monanral 

Speech 

1.000 

(p=.627) 

-0.056 
p  =  .978) 

Spatial  Speech 

■ 

-1.056 
(p  =  .595) 

Bold  blocks  indicate  significant  differences. 
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2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  M  DR  M  STRUB 
6359  WALKER  LANE  SUITE  100 
ALEXANDRIA  VA  22310 

I  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  ML  J  MARTIN 
MYER  CENTER  RM  2D3 1 1 
FT  MONMOUTH  NJ  07703-5601 

1 0  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MC  B  DAVIS 
199  E  4TH  ST  STE  C  TECH  PARK  BLDG  2 
FT  LEONARD  WOOD  MO  65473-1949 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 


1  COMMANDANT  USAADASCH 
ATTN  ATSA  CD 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARE  HR  MI  J  MINNINGER 

BLDG  5400  RM  C242 

REDSTONE  ARSENAL  AL  35898-7290 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MG  R  SPINE 
BUILDING  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARE  HRED  ARMC  FED  ELMT 

ATTN  AMSRD  ARE  HR  MH  B  STERLING 
BLDG  1467B  ROOM  336 
THIRD  AVENUE 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AVNC  FIELD  ELEMENT 
ATTN  AMSRD  ARE  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARE  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  ROOM  111 
FT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MY  M  BARNES 
2520  HEALY  AVE  STE  1172  BLDG  51005 
FTHUACHUCAAZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARE  HR  MP  D  UNGV ARSKY 
BATTLE  CMD  BATTLE  LAB 
415  SHERMAN  AVE  UNIT  3 
FT  LEAVENWORTH  KS  66027-2326 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  M  DR  B  KNAPP 
ARMY  G1  MANPRINT  DAPE  MR 
300  ARMY  PENTAGON  ROOM  2C489 
WASHINGTON  DC  20310-0300 
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ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MJK  MS  D  BARNETTE 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
SUFFOLK  VA  23435 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MQ  M  R  FLETCHER 
US  ARMY  SBCCOM  NATICK  SOLDIER  CTR 
AMSRD  NSC  SS  E  BLDG  3  RM  341 
NATICK  MA  01760-5020 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MT  DR  J  CHEN 
12350  RESEARCH  PARKWAY 
ORLANDO  FL  32826-3276 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MS  MR  C  MANASCO 
SIGNAL  TOWERS  RM  303A 
FORT  GORDON  GA  30905-5233 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MU  M  SINGAPORE 
6501  E  11  MILE  RD  MAIL  STOP  284 
BLDG  200 A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FTBENNING  GA  31905-5400 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARE  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 


ABERDEEN  PROVING  GROUND 
1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARE  Cl  OK  TECH  LIB 
BLDG  4600 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARE  Cl  OK  TP  S  FOPPIANO 

BLDG  459 


1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARE  HR  MR  F  PARAGALLO 

BLDG  459 
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